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To the Honourable Speaker of the Legislative Assembly 


I am pleased to transmit my Annual Report for submission to the Assembly in 
accordance with the provisions of section 12 of the Audit Act. 


Erik Peters, FCA 
Provincial Auditor 


Fall 2002 
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CHAPTER ONE 


Overview 





THE MANAGEMENT AND USE OF 
CONSULTING SERVICES 


This year’s report contains a number of significant concerns about the government's 
management and use of consulting services. While findings relating to consulting services 
figured in several of our value-for-money audit reports, I would like to focus on two findings 
in this overview: the first is reported in Chapter Three, Section 3.01—Ontario Works 
Program, and the second is reported in Section 3.06—Consulting Services. 


ONTARIO WORKS PROGRAM 


In 1997, the Ministry of Community, Family and Children’s Services engaged Accenture 
(formerly known as Andersen Consulting) as a consultant on its Business Transformation 
Project. This project involved a province-wide revision of the business processes for 
delivering the Ministry’s social-assistance programs and the information technology system 
supporting those processes. By March 31, 2002, Accenture had been paid about a quarter 
of a billion dollars ($246 million), out of total project costs of over $400 million, to provide 
an information-technology-based service-delivery system. As detailed in Chapter Three, 
Section 3.01, the system was seriously flawed and was characterized by its principal users as 
being, in many respects, a step back from what had been available to them previously. For 
example, the system: 


e failed in many ways to meet ministry and municipal service-manager needs; 


e made unexplained errors—for example, the system inexplicably sent about 7,000 
payments totalling $1.2 million to ineligible individuals; and 


¢ had internal control deficiencies that exposed the Ontario Works program to an 
unnecessary risk of misappropriation of funds. 


In many respects, I consider the Ministry’s involvement with Accenture to have been a very 
expensive lesson in how not to implement a new IT-based service-delivery system. Making 
the system work properly may well take much more time and will certainly take many more 
taxpayers’ dollars. The way in which the Ministry applied the common purpose 
procurement process to acquire the consulting services in this case meant that the taxpayer 
took virtually all the financial and performance risks and the consultant reaped a 
disproportionately large share of the financial rewards. 
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CONSULTING SERVICES 


We noted a dramatic increase in the government's reliance on consultants over the last five 
years. Specifically, total consulting expenditures have more than doubled since 1998 (from 
$271 million in 1997/98 to $662 million in 2001/02), with expenditures for IT consulting 
tripling (from $100 million to $313 million) and for management consulting increasing 
four-fold (from $40 million to $152 million). We therefore decided to conduct a value-for- 


money audit on consulting services at six selected ministries. 


As detailed in Chapter Three, Section 3.06, we found that the ministries frequently did not 
obtain value for money when using consulting services. Of particular concern was the 
practice of engaging consultants on a per diem basis and not on the basis of clearly defined 
deliverables at a fixed price, especially in the development of multi-million-dollar IT 
projects. As a result, the ministries assumed the risks of consultants not delivering on time 
and of having to pay the cost of missed deadlines and cost overruns. We also noted other 
obstacles frequently hindering the attainment of value for money, such as a heavy 
dependence on the use of consultants, not selecting and acquiring consulting services on a 
competitive basis, and significant weaknesses in controls over payments to consultants. As 
well, we noted that the vendor-of-record guidelines established by Management Board 
Secretariat were ambiguous in some respects and were therefore difficult for ministries to 


apply. 
Many of the obstacles noted above could have been overcome if the ministries had 


rigorously adhered to the rules set out by the Management Board of Cabinet regarding the 


acquisition and use of consulting services. 


ACCOUNTING FOR THE 
GOVERNMENT’S FINANCIAL 
ACTIVITIES 


AUDITOR’S REPORT ON FINANCIAL 
STATEMENTS 


My opinion on the financial statements for the year ended March 31, 2002, which is clear 
of any qualifications or reservations, also covers the restatement of prior years’ financial 
results presented in the province’s statements this year. This restatement, which is discussed 
further in Chapter Five, largely resulted from the provisional recording of the substantial 
tax-remittance error made by the federal government’s Canada Customs and Revenue 
Agency. 
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THE STRANDED DEBT OF 
THE ELECTRICITY SECTOR 


As discussed more comprehensively in Chapter Five of this report, in my view there is an 
increasing risk that the stranded debt of the electricity sector, which is to be recovered from 
electricity ratepayers, may ultimately become, in part or completely, a liability of Ontario’s 
taxpayers. I am of this view because the stranded debt has increased by about $700 million 
since the restructuring of the electricity sector on April 1, 1999 and because the financial 
performance of the electricity sector in the year ended March 31, 2002 was well below 
expectations and resulted in Ontario’s taxpayers absorbing $341 million of electricity sector 
costs. As well, the earliest estimated defeasance date of the stranded debt has been delayed 
by two years, from 2010 to 2012. 


In future considerations of privatizations in the electricity sector, I urge the government to 
carefully evaluate the impacts of each privatization on both the ratepayers’ ability and the 
time required to defease the stranded debt. 


MORE CONSISTENT AND INCLUSIVE 
ACCOUNTING 


As explained in more detail in Chapter Five of this report, | am pleased that the government 
has announced its intention to adopt consistent accounting rules for all financial reports by 
converting Ontario’s legislative spending control to the accrual basis of accounting. In past 
years, I urged the government to make this change so that the budget, the financial 
statements in the Public Accounts, and the Estimates (Appropriations) would all use the 
same accounting rules. In Chapter Five, I illustrate the impact of the accounting 
inconsistencies on 2000/01 health-care spending and on 2001/02 corporate tax revenue. 
The government plans to overcome this kind of inconsistency beginning in the 2003/04 


fiscal year. 


The government also announced that, effective April 1, 2003, it plans to account for 
tangible capital assets by recording them as assets and amortizing them over their useful life. 
This practice is similar to that followed in the private sector and would supersede the 
current practice of recording the assets as expenditures when they are paid for. This planned 
change represents, as outlined in Chapter Five, the implementation of an accounting 
standard of the Public Sector Accounting Board of the Canadian Institute of Chartered 
Accountants whose adoption I have been recommending for several years. 


In Chapter Five of this report, I also advocate that the government include in its financial 
reporting those financial activities of school boards, universities, colleges, and hospitals that it 
has determined to be under its control. Such an accounting should present to Ontario’s 
taxpayers the extent to which these organizations’ actual expenditures, revenues, tangible 
capital assets, other assets, liabilities, and debts are controlled by the government. 
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BETTER INFORMATION FOR 
DECISION-MAKING 


As I have emphasized in previous reports, having good information for decision-making is 
essential. Appropriate, reliable, and timely information enables decision-makers to 
accurately assess the economy, efficiency, and effectiveness of government programs and 
activities. Such information provides a critical base for decision-makers to decide whether to 
continue, discontinue, or change government programs and activities, including the use of 
alternative service delivery or common purpose procurement. Good administration of 
public funds depends on good decisions based on good information. 


One of the significant themes of my Annual Report this year, as in past years, is that 
ministries still require improvements in the quality of their information about the economy 
and efficiency of the programs and services they deliver. We found that ministries often 
lacked adequate procedures for measuring and reporting program effectiveness, especially 
for those programs funded through transfer payments. Obtaining such information would 
offer significant opportunities to improve decision making, which would lead to an 
improved administration of public funds. 


Many services paid for by government are being provided by service-delivery agents, and 
the government does not have access to information about the day-to-day operations of such 
agents. Since about half of our tax dollars go to these agents in the form of transfer 
payments, it is vital that the government know the extent to which they are achieving 
intended results and whether or not taxpayers are receiving value for money spent. The key 
to obtaining this knowledge is ensuring the availability of appropriate, reliable, and timely 
information. 


TOWARDS BETTER ACCOUNTABILITY 


As in previous reports, Chapter Two of this report highlights the importance of enacting my 
Office's long-standing proposed amendments to the Audit Act and the continuing need for 
a legislated accountability framework for the broader public sector. Chapter Two also 
discusses the Ontario Innovation Trust—specifically, the inadequacy of its accountability to 
the Legislature for the large sums of money transferred to it. 


VALUE-FOR-MONEY AUDIT REPORT 
SUMMARIES 


The following are summaries of the 11 value-for-money audits reported on in Chapter 


Three of this Annual Report. 
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3.01 Ministry of Community, Family and Children’s Services 
Ontario Works Program 


Under provisions of the Ontario Works Act, the Ontario Works program of the Ministry of 
Community, Family and Children’s Services provides employment and temporary financial 
assistance to individuals on condition that they satisfy requirements intended to help them 
find and maintain paid employment. For the 2001/02 fiscal year, the Ministry’s share of 
financial assistance provided to individuals was approximately $1.4 billion. The Ministry's 
share of costs for program administration was $171 million. 


Since 1997, the Ontario Works program has been subject to a much needed and complex 
Business ‘Transformation Project. (We previously reported on this project in our 1998 and 
2000 reports.) This Project included the engagement of Accenture (formerly Andersen 
Consulting) to develop a new service delivery system under a Common Purpose 
Procurement (CPP) agreement. The intent of CPP was for the Ministry to work closely with 
the selected private-sector vendor to develop and implement new ways of delivering services 
and, in so doing, share the investment in and risks and rewards of the project. However, we 
concluded that the Ministry did not meet this objective in that it accepted most if not all of 
the risk for the Business ‘Transformation Project while Accenture received a disproportionate 
amount of the rewards. Specifically we found: 


e As of March 2002, the Ministry had paid Accenture $246 million, which is 
significantly more than the $180-million payment cap agreed to. 


e The savings attributed to the Business Transformation Project and hence to Accenture 
were exaggerated. 


e We reviewed the new service delivery system, which was fully implemented in January 
2002, and we shared the view of municipal service manager staff that the system was in 
many respects a step back from what had previously been available to them, that it had 
been inadequately tested, and that it was not a finished product at the time of its release. 


¢ Our own testing found that the new service delivery system had numerous unresolved 
defects, such as failing to provide certain needed information and providing information 
that was often inaccurate or in a form that was not useful. There were unexplained 
errors—for example, benefit payments totalling $1.2 million were sent to ineligible 
individuals—and there were significant internal control deficiencies. 


With respect to the administration of the Ontario Works program, we concluded that the 
Ministry had little assurance that only eligible individuals received the correct amount of 
financial assistance. The primary reason for this was that ministry requirements for 
municipal service managers to determine recipient eligibility for financial assistance and to 
provide that assistance in the correct amount were often not met. For example, in the case of 
one of the service managers that we visited, 95% of the files we reviewed lacked at least one 
of the information requirements necessary to establish eligibility and to ensure the correct 
amount of assistance 1s paid. 
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We made recommendations for improving program delivery and received commitments 
from the Ministry that it would take corrective action. 


3.02 Ministry of Finance 
Corporations Tax 


Generally, the Ontario Corporations Tax Act imposes taxes on all corporations that have a 
permanent establishment in Ontario or that owned and received income from or disposed 
of real property in Ontario. For the 2001/02 fiscal year, the province recorded 
approximately $6.6 billion in corporations taxes ($9.2 billion for the 2000/01 fiscal year). 
The tax collection effort was carried out by 770 staff and cost about $45 million, of which 
90% was for salaries and benefits. 


We concluded that where corporations did not voluntarily comply with the provisions of 
the Ontario Corporations Tax Act, the Ministry did not have adequate policies and 
procedures in place to ensure that the appropriate amount of corporations tax was being 
declared and remitted by taxpayers in accordance with statutory requirements. It is our view 
that, as a result, the tax gap with respect to provincial corporations tax—that is, the 
difference between the amount of corporations tax actually collected and the amount that 
should be collected—may well be substantial. In this regard, we found that the Ministry did 
not assess or evaluate the extent to which the overall tax gap affected provincial corporations 
tax revenue, or the collection effort. 


We noted an increase in the extent to which corporations did not voluntarily comply; of the 
763,000 corporations with active accounts on the Ministry’s tax roll, 355,000 
corporations—or one in two—did not file required returns. In 1996, at the time of our last 
audit, about one in five corporations did not file required returns. 


We also noted that the Ministry did not regularly compare all active registrants in the 
Ministry of Consumer and Business Services’ (MCBS) database with those on the 
corporations tax roll to ensure that all corporations that are registered with MCBS and are 
required to file a tax return continue to be included in the corporations tax roll. 


With respect to its function of auditing corporations tax returns, we found that for 
corporations with annual gross revenues of $500,000 and over, the number of desk audits 
completed was about half of the number planned. For the corporations that have gross 
revenues under $500,000, which represent about 87% of the total number of corporations 
on the tax roll, very few field or desk audits were performed. Although the Ministry has 
made a deliberate decision to rely on the Canada Customs and Revenue Agency for the 
audit of smaller corporations, we noted that it had not obtained the necessary information 
to assess whether such reliance is justified. 


We made a number of recommendations for improvement and received commitments 
from the Ministry that corrective action would be taken. 
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3.03 Ministry of Health and Long-Term Care 
Community Mental Health 


Through its Integrated Health Care Program, the Ministry provides transfer payments to 
community agencies or general hospitals to deliver community-based mental health 
programs and to help cover the costs for sessional fees, homes for special care, and other 
housing with supports for individuals with mental illness. During the 2001/02 fiscal year, 
the Ministry provided approximately $390 million in transfer payments for community- 
based mental health services. The Ministry estimated that approximately 2.5% of the 
population of Ontario, or 300,000 people, are seriously mentally ill. 


We concluded that many of the fundamental issues and concerns identified in our audits 
over the last 15 years had not been comprehensively addressed. In particular, except in the 
case of assertive community treatment teams, the Ministry still had not clearly defined its 
expectations for community mental health. We also found that: 


e In many areas of the province there is still no comprehensive source of information 
about available mental health services or how to access those services. In addition, there 
is minimal co-ordination among agencies providing services. 


e The Ministry did not have sufficient information to enable it to assess whether mentally 
ill people were adequately cared for and whether funding provided for community- 
based mental health services was being prudently spent. 


¢ The Ministry was not tracking the number of people receiving or waiting for 
community mental health services or the waiting times to access services. This limited its 
ability to assess whether there were sufficient and appropriate resources to meet the 


needs of the seriously mentally ill. 


e The Ministry had not determined the number or type of housing spaces required to 
meet the needs of seriously mentally ill individuals or whether existing housing was 


meeting the needs of the individuals already housed. 


Also, the Ministry had not given sufficient consideration to the funding of community 
mental health agencies based on an assessment of the number of patients requiring services 
and the complexity of patients’ needs. 


¢ In the seven regions of the province, annual per capita funding for community mental 
health services ranged from $11 to $60. The funding was primarily historically based, 
rather than being based on the relative need for services and the costs of delivering 
services in different regions of the province. Funding based on assessed need helps 
ensure that individuals with similar needs have access to similar services regardless of 
where they live in the province. 


¢ Since 1992, there have been no increases in base funding provided to community 
mental health agencies for programs that were operating at that time. One district 
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health council noted that this forced community mental health agencies “to reduce 
services to the seriously mentally ill in order to stay within existing base budgets.” 


We made a number of recommendations for improvement and received commitments 
from the Ministry that it would take action to address our concerns. 


3.04 Ministry of Health and Long-Term Care 
Long-Term Care Facilities Activity 


Long-term-care facilities provide care and services to individuals who are unable to live 
independently at home and require the availability of round-the-clock nursing service to 
meet their daily nursing and personal care needs. These facilities comprise nursing homes 
and homes for the aged. 


The Ministry’s key responsibility regarding the operations of long-term-care facilities is to 
ensure that they are delivering services to residents in accordance with their service 
agreements with the Ministry and in compliance with applicable legislation and ministry 
policies. For the 2001/02 fiscal year, long-term-care facilities received approximately 
$1.6 billion in funding from the Ministry and approximately $793 million in 


accommodation payments from residents. 


We concluded that, in certain significant respects, the Ministry did not have all of the 
necessary procedures in place to ensure that long-term care resources were managed with 
due regard for economy and efficiency and that long-term care facilities were complying 
with applicable ministry policies. A number of our concerns were also reported on in our 
1995 Annual Report. Our main concerns were as follows: 


¢ The Ministry had not developed facility staffing standards or models for staff mixes for 
providing quality care. Accordingly, the Ministry did not have a sufficient basis for 
determining appropriate levels of funding. 


¢ The Ministry had not addressed the results of a 2001 consulting report that noted that 
residents of Ontario’s long-term care facilities received fewer nursing and therapy 
services than those in similar jurisdictions with similar populations. 


¢ Although the Ministry inspected all long-term-care facilities in 2001, it did not adjust 
the depth of its inspections for facilities with a history of failing to meet ministry quality 
standards. We also noted that, contrary to legislation, none of the nursing homes in 
Ontario had current ministry-issued licences at the time of our audit. At least 15% of 
licences had expired more than one-and-a-half years ago. As well, most nursing homes 
that opened after 1998 had never been issued a licence. 


* The Ministry was not adequately tracking complaints, unusual occurrences, and 
outbreaks of contagious diseases to identify and resolve systemic problems. 


* Surplus funds were not being recovered from facilities on a timely basis. Ministry delays 
in completing reconciliations for the 1999 calendar year resulted in approximately 
$5 million in interest expenses being passed on to the taxpayers. 
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We also concluded that the Ministry’s procedures for providing accountability to the public 


and ensuring that facilities provide services efficiently and effectively were impaired by: 


¢ insufficient financial information from facilities to allow the Ministry to determine 
whether funds had been used in accordance with the Ministry’s expectations; and 


° the lack of outcome measures to address the appropriateness of services provided, 
including the quality of care received by residents. 


Through its long-term-care redevelopment project, the Ministry allocated funding to build 
new long-term-care facilities containing approximately 20,000 new beds to regions of the 
province where the need for additional beds was the greatest. The Ministry was also 
providing financial assistance to ensure existing facilities meet minimum structural and 
environmental standards. However, the Ministry did not have a process in place for 
periodically reviewing whether its target of 100 beds per 1,000 individuals aged 75 and 


over was appropriate. 


We made a number of recommendations for improvement and received commitments 
from the Ministry that it would take action to address our concerns. 


3.05 Management Board Secretariat 
Electronic Service Delivery 


Many governments, including Ontario, are increasingly using electronic means to provide 
information about government services to individuals and businesses, as well as to deliver 
some of those services. This method of providing services is known as electronic service 
delivery (ESD). Through ESD, the government is organizing and integrating services 
through call centers, interactive voice response systems, Web sites, e-mail, fax, CD-ROM, 
public access terminals and kiosks, and electronic payment systems. 


In June 2000, the Management Board of Cabinet approved a government-wide ESD 
strategy aimed at improving the quality of service to Ontarians and businesses by providing 
client-focused, integrated, accessible, and cost-effective government services electronically. 
The government plans to have over 80% of the services it delivers available through 
electronic means by 2003. The Management Board Secretariat (MBS) is responsible for the 
government's ESD strategy. 


The government has set ambitious targets for ESD and is committed to increasing 
“Ontarians’ satisfaction with government services by becoming a world leader in delivering 
services on-line” by 2003. Although significant strides have been made in implementing 
ESD to date, we concluded that the government will likely fall short of meeting its ESD 
targets if it does not accelerate the pace of ESD implementation. A more proactive and 
hands-on central management of the ESD initiative is needed. In addition, we noted that 
communications efforts to promote ESD have been insufficient to increase public awareness 
and use of services delivered electronically. 
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We reviewed four high-impact service delivery projects at the ministries we visited and noted 
that there was room for improvement with respect to a number of security and service 


availability issues. 


We made recommendations for improvements in each of these areas and received 
commitments from the Ministry that the necessary corrective actions would be taken. 


3.06 Management Board Secretariat and the Ministries of the 
Environment, Finance, Health and Long-Term Care, 
Natural Resources, and Public Safety and Security 


Consulting Services 


Consulting services, as defined under the Management Board of Cabinet Directive on 
Consulting Services (Directive), are services provided for a fee, on the basis of a defined 
assignment, and relating to management consulting, information technology (IT) 
consulting, technical consulting, and research and development. 


Over the past five years, there has been a substantial increase in annual consulting services 
expenditures at Ontario ministries, from $271 million in 1998 to $662 million in 2002. 
Our audit encompassed the following six selected ministries (the Ministries): Management 
Board Secretariat (MBS), Environment, Finance, Health and Long-Term Care, Natural 
Resources, and Public Safety and Security (Public Safety and Policing Services Divisions, 
constituting the former Ministry of the Solicitor General). For the. 2001/02 fiscal year, these 


ministries incurred $293 million in consulting services expenditures. 


Our audit concluded that, in many respects, consulting services were not acquired and 
managed with due regard for value for money. The following is a summary of our major 
concerns: 


¢ There was a heavy dependence on the use of consultants. Hundreds of consultants were 
engaged at per diem rates that were on average two to three times higher than the 
salaries of ministry employees performing similar duties. For instance, over half of the IT 
workforce at the Ministry of Public Safety and Security was made up of consultants, 
including 40 former ministry employees who, within a few days of having left the 
Ministry, returned at per diem rates that were more than double their salaries as 
employees. 


¢ The Ministries often awarded continuous agreements to the same consultant with little 
or no change to the original deliverables. For example, a consultant engaged by the 
Ministry of Health and Long-Term Care on a six-month assignment at $96,000 was 
awarded successive contracts that extended the term to two years and resulted in a total 
cost of $360,000. The consultant was eventually replaced with a full-time staff member 
at an annual salary of approximately $60,000 
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In the development of multi-million-dollar IT projects, the Ministries often engaged 
consultants on a per diem basis to do the work instead of calling for open tender. This 
lack of open tendering did not ensure that the most qualified consultants were acquired 
at the best available price and that all suppliers of consulting services were given fair 
access and treated in an open and transparent manner. In addition, by compensating 
consultants on a per diem basis and not on the basis of a fixed price and fixed 
deliverables, the Ministries assumed the risk and cost of consultants not delivering their 
work on time, even when such problems may have been caused by unsatisfactory 
performance and inefficiencies on the part of the consultants. 


A forensic accounting firm was engaged by MBS to review certain real-estate 
transactions entered into by the Ontario Realty Corporation. While its estimated fees 
ranged from $150,000 to $500,000, the firm was paid almost $6 million for expanded 
work, and a new contract was not entered into to reflect the revised scope and objectives 
of the ongoing investigative work. 


The Ministries frequently did not ensure that consultants’ provincial taxes were in good 
standing. At MBS, two consultants in our sample had tax arrears of approximately 
$110,000 and $35,000 respectively. At the Ministry of Public Safety and Security, two 


consultants who received contracts were in default for not filing corporate tax returns. 


There were significant weaknesses in controls over payments to consultants. For 
example: 


- The Ministry of the Environment and the Ministry of Public Safety and Security 
made numerous payments to consultants that exceeded the ceiling price of 
contracts. We found no evidence of prior approvals by the Deputy Minister or 
designate as required by the Directive. 


- At MBS, one consultant’s rates were permitted to increase significantly, from 
$725/day in April 2000 to $1,800/day in May 2000 and to $2,600/day in 
September 2000, without documented rationale for these large increases. As well, 
one consulting firm was reimbursed for meal charges at ten times the rates allowed 
to government employees. 


- At the Ontario Realty Corporation (ORC), one consulting firm was paid over 
$1 million even though it only had written agreements with ORC that together had 
a billing maximum of $210,000. 


We made a number of recommendations for improvement and received commitments 
from all the Ministries that they would take corrective action. 


3.07 Ministry of Natural Resources 


Ontario Parks Program 


The Ontario Parks Program of the Ministry of Natural Resources is responsible for 
managing provincial parks and protected areas in support of the Ministry's vision of 
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sustainable development of natural resources and its mission of managing such resources for 
ecological sustainability. The primary objectives of the Program are to protect natural 
resources, provide recreational opportunities, develop tourism, and enhance appreciation of 
the province’s natural and cultural heritage. At the time of our audit there were 277 
provincial parks covering over 70,000 km’. 


Overall, we concluded that, in many respects, the Ministry did not ensure compliance with 
the legislation and policies designed to ensure the sustainable use and development of park 
resources and that the Ministry did not have adequate procedures in place to measure and 
report on the effectiveness of the Program. In addition, we noted a number of instances 
where procedures to ensure due regard for economy and efficiency needed to be improved. 


Specifically, we observed the following: 


¢ The enforcement activity that was carried out was inadequate in that over 70% of park 
superintendents indicated that parks were not being effectively patrolled. As a result of 
the Ministry’s not meeting its protection mandate, natural resources had been adversely 
affected and in some cases destroyed. 


¢ The Ministry had management plans in place for only 117 of the 277 provincial parks. 
Such plans are essential if animal and plant life resources are to be managed and 
protected. We noted instances where inadequate planning and a lack of action resulted 
in uncontrolled wildlife growth and habitat destruction that threatened the 
sustainability of other species. 


¢ The Ministry did not have an overall strategy in place to manage species at risk of 
extinction in the province even though the Endangered Species Act has been in force 
since 1971. Of the 29 species deemed by regulation to be at risk, only five had recovery 
plans in place. Three species that did not have recovery plans in place can no longer be 
found in Ontario. 


* Customer service standards were not met for the parks’ Computer Reservation and 
Registration Accounting System that was operated by a private service provider. Over 
65% of our sample telephone calls were not answered either because of a busy signal or 
because we were put on hold for 15 minutes, after which time we hung up the phone. 


We made recommendations for improvements in each of these areas and received 
commitments from the Ministry that it would take corrective action. 


3.08 Ministry of Public Safety and Security 
Community Services Program 


The Ministry's Community Services Program is responsible for supervising all adult 
offenders (18 years of age and older) and young offenders (16 to 17 years of age) who are 
under some form of conditional release—that is, who are on probation, serving a 
conditional sentence, or on parole. The objectives of the Program are to protect the public 
by monitoring offenders in the community and to rehabilitate offenders through training, 
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treatment, and services that afford them opportunities for successful personal and social 
adjustment in the community. 


On any given day, there is an average of 65,000 offenders being supervised by the Ministry 
in the community. Of these offenders, 95% were on probation, 4% were serving 
conditional sentences, and the remaining were on provincial parole. 


At March 31, 2002, the Ministry employed approximately 770 probation and parole 
officers throughout the province. In addition, as part of the Program, the Ministry contracts 
with selected community agencies to provide a variety of counselling and treatment 
programs. In 2001/02, total program expenditures amounted to approximately 

$82 million. 


Since our last audit in 1995, the Ministry, in 1999, initiated a new offender management 
model, which highlights offenders’ correctional needs that should be addressed to effectively 
reduce the risk of offenders reoffending. While we acknowledged that the Ministry was in 
the process of implementing this new model, we concluded that there were a number of 
deficiencies in its procedures that hindered the effective supervision of offenders in the 
community. For instance: 


e At the offices we visited, over 40% of offenders who had committed additional “level I” 
offences while under ministry supervision lacked a risk and needs assessment and/or a 
management plan. (Level I offences include sexual assault, assault causing bodily harm, 
uttering death threats, and other violent crimes.) 


e At the five offices we visited, of the cases involving level I offenders who later committed 
additional offences while under supervision, we noted that over 30% had not been 
followed up on a timely basis after the offender failed to comply with the conditions of 
their supervision. 


e We estimated there were approximately 10,000 arrest warrants outstanding for 
offenders in the community, including some that had been issued as far back as 10 
years. Many of these offenders were assessed as high risk and had committed serious 
offences, such as sexual assault and assault causing bodily harm. The Ministry did not 
know how many of the offenders against whom there were arrest warrants outstanding 
were still at large. 


While we recognize that once a warrant is issued, the police—not ministry staff—are 
responsible for apprehending the offenders, the Ministry and the police need to work 
more closely together so as not to expose the community to significant risk. 


¢ According to a ministry report, correctional programs for offenders were often not 
available in their local community. For example, of the over 3,000 sex offenders being 
supervised by the Ministry, fewer than 600 received appropriate rehabilitation 
programs. 
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We made a number of recommendations for improvement and received commitments 
from the Ministry that it would take corrective action. 


3.09 Ministry of Public Safety and Security 
The Ontario Parole and Earned Release Board 


The Ontario Parole and Earned Release Board (Board) makes decisions about parole for 
offenders sentenced to less than two years of imprisonment. Offenders are eligible for parole 
upon serving one-third of their sentences. Offenders that are granted parole serve the full 
length of their sentences (one-third in an institution followed by two-thirds in the 
community under supervision and conditions set by the Board); offenders that are not 
eranted parole are released from an institution after serving two-thirds of their sentences. 
Effectiveness in contributing to the safety of society requires the Board to help more low-risk 
offenders successfully reintegrate into the community by controlling the timing and 
conditions of their release. 


For the 2001/02 fiscal year, the Board had four full-time and over 40 part-time members 
with total expenditures of approximately $3 million. In the 2000/01 fiscal year, there were 
approximately 2,100 parole hearings, of which 28% resulted in parole being granted. 


We concluded that the Board’s mandate of protecting society by effectively reintegrating 
offenders into the community was hindered by a dramatic reduction in the number of 
eligible inmates being considered for parole. The decline in the number of hearings from 
6,600 to 2,100, combined with a steady drop in parole grant rates from 59% to 28%, has 
resulted in fewer than 600 inmates being granted parole in 2000/01, as compared to 3,800 
in 1993/94. 


According to board studies, factors contributing to this decline included inmates not 
receiving the required parole information and inmates waiving parole hearings because they 
felt there was little chance of getting a fair and unbiased hearing. As well, significant 
numbers of offenders were denied the opportunity to have their cases heard as a result of 
widely differing practices among different regions. 


In addition, we found that although Ontario’s parole grant rates have significantly declined 
since 1993/94, its rates of parolees re-offending during parole have been generally higher 
since that same time. This situation requires research by the Board to determine what 
further action, if any, is required. 


We also noted that: 


e The Board often did not obtain all relevant information before rendering parole 
decisions, nor did it record the rationale for its decisions to not impose special conditions 
that were recommended by parole officers or police. 


* The Board set performance goals for 2001/02 that were below those already achieved; 
thus, its goals do not serve to encourage an improvement in board performance. 
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¢ Ontario had no formal selection process to assess the abilities, skills, commitment, and 
suitability of potential board members, nor did the Board have the opportunity to 


provide input on the initial screening of potential candidates. 


We made a number of recommendations for improvement and received commitments 
from the Board that it would take corrective action. 


3.10 Ministry of Tourism and Recreation 
Tourism Program 


The Ministry of Tourism and Recreation estimates that the tourism industry employs 
approximately 500,000 people and generates $17 billion for the Ontario economy. The 
Ministry's Tourism Program (Program) is responsible for promoting tourism in Ontario. The 
role of the Ontario Tourism Marketing Partnership Corporation (Corporation), a ministry 
agency, is to market Ontario as a tourist destination. For the 2001/02 fiscal year, tourism 
operating expenditures totalled $83 million, of which $52 million was spent by the 
Corporation. 


Ministry statistics indicate that the number of tourists visiting Ontario has gradually 
declined over the past 10 years. The decline results from a 25% decrease in the number of 
domestic tourist visits (including those made by Ontario residents), which was partially offset 
by an increase in international tourist visits. 


We concluded that the Ministry and the Corporation did not have sufficient procedures to 
measure and report on the effectiveness of the Program in promoting tourism in Ontario. 
In its annual business plan, the Ministry did not report the actual results achieved for any of 
its previously published performance measures, and, after three years of operation, the 
Corporation had not submitted an annual report to the Legislature as required. 


Given the significance of tourism to the Ontario economy, we also concluded that the 
Ministry needs to take a leadership role in developing a long-term tourism strategy to help 
co-ordinate the activities of the many organizations that contribute to the promotion of 
tourism in the province. We also found that the Ministry and the Corporation did not have 
adequate procedures in place to ensure that the Program was delivered with due regard for 
economy and efficiency. Specifically, we noted the following: 


e There was no process in place to collect information on the tourism-related activities 
undertaken by other ministries or on the financial support provided by other ministries 
to the tourism industry. As a result, there was a risk of overlap and duplication of 
tourism-related programs and services. 


¢ ‘Tourism publications were not sufficiently comprehensive and were not published on a 
timely basis. For example, the Corporation’s main tourism guide lists only 1,400 of an 
estimated 8,000 tourism facilities, and the 2001/02 winter events guide was not 
published until December 2001 and contained listings for events that had already taken 
place. 





Overview 1p 


| Chapter One | 





Chapter One 





¢ The promotion of accommodation rating systems in Ontario has not been adequately 
co-ordinated between the ministries and the private sector. As a result, Ontario is one of 
only a few leading tourist destinations that does not have province-wide quality 
standards. 


¢ In many instances, the acquisition of management consulting services was not justified 
by a business case, nor were related contracts signed on a timely basis. Moreover, several 
contracts were awarded directly to the vendor without competition, and other projects 
were split into separate contracts, thus allowing the Ministry to avoid open competition 
requirements. 


We made recommendations for improvements in each of these areas and received 
commitments from the Ministry that it would take corrective action. 


3.11. Ministry of Training, Colleges and Universities 

Training Division 
The mandate of the Training Division (Division) of the Ministry of Training, Colleges and 
Universities is to set standards for employment services and adult literacy, to help employers 
develop a skilled workforce to stay competitive, and to provide leadership on labour-market 
and training issues. [he Division’s programs and services are intended to assist both 
individuals and employers in increasing skill levels and to help individuals make the 
transition from unemployment to employment and from education and training to the 
labour force. 


Division expenditures for the 2001/02 fiscal year totalled $346.3 million. Our audit 
focused on the following major programs: Job Connect; Summer Jobs Service; 
Apprenticeship; and Literacy and Basic Skills. These programs account for approximately 
75% of the Division’s expenditures. 


Colleges of Applied Arts and Technology (community colleges), school boards, and 
community-based, not-for-profit organizations form the network of agencies responsible for 
delivering three of the major transfer-payment programs: Job Connect; Summer Jobs 
Service; and Literacy and Basic Skills. Employers are the primary deliverers of 
apprenticeship training, while community colleges and private training institutions that are 
funded by the Ministry and the federal government deliver in-school training assistance. 


We concluded that the systems and processes necessary to ensure that services are delivered 
economically and efficiently and that the programs’ objectives and expected outcomes were 
being achieved had not been completed. Some of our specific observations included: 


¢ Efforts to co-ordinate enforcement responsibilities and share information with the 
Ministry of Labour and other bodies responsible for workplace inspections have not 
been sufficient to determine the extent to which uncertified individuals are working in 
restricted trades. Effective enforcement of restricted trades is necessary to ensure 
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legislated objectives for protecting public and workplace safety are met and to maintain 


the value of obtaining certification in restricted trades. 


¢ The Ministry was not monitoring the quality of apprenticeship training provided by 
employers and in-school training providers. 


¢ The Ministry had not linked funding for providers of in-school apprenticeship training 


and for delivery agencies of Literacy and Basic Skills services to their performances in 
providing quality training. The systems and procedures needed to collect and report 
meaningful performance information were under development. 


We also found that the Ministry did not adequately control the acquisition and 
management of about $11 million worth of consulting and other services that were 


acquired on the Ministry’s behalf through not-for-profit agencies over the last several years. 


In particular, the Ministry did not adhere to prudent purchasing practices and did not 
obtain the approvals from the Minister and Management Board of Cabinet that would 
have been required if the projects had been undertaken and the services acquired by the 
Ministry directly. Our findings included that: 


e Services amounting to about $8 million were acquired from private-sector suppliers 
with little or no competition. 


¢ GST charges totalling $600,000 were incurred because the agencies were not GST- 
exempt—$235,000 of that amount was overbilled and should be recovered. 


We made a number of recommendations for improvement and received commitments 
from the Ministry that it would take corrective action. Some corrective actions were 


underway at the time of our audit. 
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CHAPTER TWO 


Towards Better 
Accountability 





In all previous years, I have used Chapter Two of my report to address specific issues of 
governance and accountability in government. Again this year, | am bringing forward issues 
that warrant discussion to improve accountability to the Legislature for the prudent use of 


public funds. ‘These are: 
¢ legislative proposals for better public-sector accountability; and 


¢ continued concerns regarding public accountability for the Ontario Innovation Trust. 


LEGISLATIVE PROPOSALS TO 
IMPROVE PUBLIC-SECTOR 
ACCOUNTABILITY 


STATUS OF RECOMMENDATIONS FOR 
AMENDMENTS TO THE AUDIT ACT 


In my 2001 Annual Report, I was pleased to note the government's intention to amend the 
Audit Act as announced in its Speech from the Throne on April 19, 2001. The 2001 
Throne Speech announcement to amend the Audit Act was intended to permit the 
Provincial Auditor to assess the extent to which organizations funded by Ontario taxpayers 
use that money prudently, effectively, and for the purposes intended. Inconclusive 
discussions on a comprehensive set of amendments took place during the summer and fall 
of 2001 between my Office and the former Minister of Finance, the Honourable Jim 


Flaherty, and his staff. 


I want to reiterate that I am of the view that all transfer-payment partners should be subject 
to legislated public accountability, including performance reporting and a full-scope 
legislative audit regime, because provincial monies flowing to grant-recipient organizations 
continue to represent the single most significant demand on the province’s financial 
resources, with about 50% of total government expenditures flowing to grant-recipient 
organizations. Under the current Audit Act, the Provincial Auditor may carry out only 





18 2002 Annual Report of the Provincial Auditor of Ontario 





limited-scope audits of grant recipient organizations by examining their accounting records 
to determine whether the grants were used for the intended purposes. This limitation on 
access to information prevents my Office from conducting full-scope value-for-money 
audits of grant-recipient organizations. 


For the past 11 years, my Office has, with the support and repeated recommendations of 
the Standing Committee on Public Accounts, pursued amendments to the Audit Act. These 
amendments would provide the Provincial Auditor with the discretionary authority to 
perform full-scope value-for-money audits of organizations that derive a very significant 
portion of their income from provincial grants—such as school boards, hospitals, 
community colleges, and universities. 


More recently, the major thrust of my proposed amendments to the Audit Act have been 
included in Bill 5, entitled An Act to amend the Audit Act to insure greater accountability of 
hospitals, universities and colleges, municipalities and other organizations which receive grants 
or other transfer payments from the government or agencies of the Crown. Bill 5 was introduced 
on April 23, 2001 by Mr. John Gerretsen, Chair of the Standing Committee on Public 
Accounts. The Bill passed second reading on June 20, 2002 and was referred to the 
Standing Committee on Public Accounts for detailed review. 


It is expected that the Standing Committee on Public Accounts will hold hearings on Bill 5 
in fall 2002. | 


THE PROPOSED PUBLIC-SECTOR 
ACCOUNTABILITY ACT 


Since 1993, I have advocated a legislated accountability framework for the broader public 
sector. In 1993, the Standing Committee on Public Accounts, by way of a unanimous 
motion, supported my pursuit of a workable legislated accountability framework. As 
reported in our 1993 Annual Report, the following are two of the primary reasons for 
advocating such an accountability framework: 


¢ The Legislative Assembly and the ministers need better legislated or regulatory tools to 
establish greater accountability for the economy, efficiency, and effectiveness of program 
delivery. A framework that is legislated becomes a tool of the Legislative Assembly, of the 
ministers, and of the funding ministries and agencies that can be used to help ensure 
cost-effective program and service delivery. 


¢ Ministry management and recipients of transfer payments need a framework to hold 
them accountable for the economy, efficiency, and effectiveness of their activities. Such a 
framework would also enhance the effectiveness of my Office’s work. 


Following through on an announcement from the 1997 Ontario Budget to establish a 
legislated accountability framework to improve accountability in the public sector, the 
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government announced in its Speech from the Throne on April 19, 2001—under the 
heading “Holding the Broad Public Sector Accountable to Taxpayers” —that it would be 
introducing sweeping reforms to ensure that all public-sector institutions are accountable to 
the citizens of Ontario. In this regard, the 2001 Ontario Budget contained a proposal to 
introduce a new Public Sector Accountability Act, which would require all major public- 
sector organizations that receive taxpayer dollars from the government to report annually on 
their performances and present annual business plans and to balance their budgets every 
year. 


The government's proposal also addresses one of the recommendations of the Ontario 
Financial Review Commission in its 2001 report entitled Raising the bar: Enhanced 
accountability to the people of Ontario. Having served as Special Advisor to the Commission, 
I was encouraged by the Commission’s recommendation that the government introduce 
legislation incorporating an accountability framework. 


On May 9, 2001, the then Minister of Finance introduced for first reading Bill 46, An Act 
respecting the accountability of public sector organizations. However, Bill 46 has not advanced 
on the legislative calendar. 


CONCLUSION 


To facilitate good performance and accountability and to ensure that taxpayers’ money is 
spent prudently and for the purposes intended, both a legislated accountability framework 
and a full value-for-money legislative audit regime need to be applied to the broader public 
sector. I again strongly urge the government to enact my proposed amendments to the 
Audit Act and the proposed Public Sector Accountability Act. Once approved by the 
Legislative Assembly, I look forward to applying the amended Audit Act to a legislated 


accountability framework. 


On behalf of the Legislative Assembly, I will continue to advocate for improvements in 
public-sector accountability and will also continue to monitor government initiatives and 
developments in this important area. 


ONTARIO INNOVATION TRUST 


Since 1999, the government has transferred $750 million to the Ontario Innovation Trust 
(Trust). The Trust was established during the 1998/99 fiscal year to be an arm’s-length 
entity for the purpose of providing funding to increase the capability of Ontario universities, 
colleges, hospitals, and other non-profit organizations to carry out scientific research and 


technology development. 


In my past three Annual Reports, I have raised accountability concerns about the Trust, 
including: 
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¢ the inability of the government and the Legislature to obtain assurance that the Trust is 
spending public funds prudently and for the purposes intended and to take corrective 
action if it is not; 

° the lack of ministerial accountability to the Legislative Assembly for the Trust’s activities; 
and 


° the fact that, as the province’s Legislative Auditor, I am not permitted, under my current 
mandate, to conduct value-for-money audits of the Trust or inspection audits of the 
beneficiaries of the Trust’s grants. 


In addition to these accountability concerns, I also commented in my past three reports 
about the consequences and the appropriateness of the accounting treatment of the 
endowments to the Trust. Although technically in accordance with accounting rules 
established by the Canadian Institute of Chartered Accountants for government, the timing 
of the grant approvals allowed the government to recognize the expenditures in the years 
before the Trust actually disbursed funds to the intended grant recipients. This resulted in a 
significant exaggeration in the amounts that were reported as having been spent on 
innovation in the 1998/99 and 1999/2000 fiscal years. In fact, at the end of the Trust’s first 
year of operation—March 31, 2000—the accounts of the province gave the impression 
that the government had spent $750 million (the total flowed to the Trust from the 
province) on innovation expenditures, when only $2.5 million had actually been disbursed 
for eligible projects in that year. The accounting treatment applied also allowed the 
government to distort financial results in the first year and in subsequent years. 


After the Trust’s first year of operation, I concluded that a significant amount of the 

$750 million was preflowed to the Trust well before it actually required the funds. For the 

three-year period ending on March 31, 2002, the Trust had only disbursed $240 million 

for eligible projects, and it held $577 million of taxpayers’ funds without an accountability 
requirement to the Legislative Assembly. 


The 2002 Ontario Budget included a $300 million enhancement to the Trust. This 
additional commitment to the Trust will, when paid, bring the government's total transfer to 
the Trust to over $1 billion. 


I continue to hold the view that all transfer-payment partners should be subject to legislated 
public accountability, including performance reporting and a better legislative audit regime. 
This would permit the Legislature to evaluate what was accomplished with the funding 


provided and to ensure that any necessary corrective action is being taken. 


During the past audit year, the Standing Committee on Public Accounts held a meeting 
with the officials of the Trust to review my past commentary. [he Committee acted on my 
concerns and, in fall 2002, tabled in the Legislature its recommendations to improve public 
accountability and provide for a better legislative audit regime for the Trust. 
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CHAPTER THREE 


Reports on 
Value-for-money (VFM) 
Audits 





Our value-for-money audits are intended to examine how well the government’s programs 
and activities are being managed and whether they comply with relevant legislation and 
policies and, where appropriate, to identify opportunities for improving on the economy, 
efficiency, and effectiveness of their operations. These audits are conducted in compliance 
with subsection 12(2) of the Audit Act, which requires the Office to report on any cases 
observed where money was spent without due regard for economy and efficiency or where 
appropriate procedures were not in place to measure and report on the effectiveness of 
programs. This chapter contains the conclusions, observations, and recommendations for 
the value-for-money audits conducted in the past audit year. 


Because of the size and complexity of the province's operations and administration, it is 
impossible to audit each program every year. Instead, the Office selects which programs and 
activities to audit on a cyclical basis, so that all major programs and activities are audited 
about every five years. The programs and activities audited this year were selected by the 
Office's senior management based on various criteria, such as a program’s financial impact, 
its significance to the Legislative Assembly, related issues of public sensitivity and safety, and 
the results of past audits of the program. 


We plan, perform, and report on our value-for-money work in accordance with the 
professional standards for assurance engagements, encompassing value for money and 
compliance, established by the Canadian Institute of Chartered Accountants. 


Before beginning an audit, our staff meet with auditee representatives to discuss the focus of 
the audit. During the audit, staff maintain an ongoing dialogue with the auditee to review 
the progress of the audit and ensure open lines of communication. At the conclusion of the 
audit field work, a draft report is prepared, reviewed internally, and discussed with the 
auditee. The Provincial Auditor and senior office staff meet with the deputy minister or 
agency head to discuss the final draft report and to finalize the management responses to 
our recommendations, which are then incorporated into each of the VEM sections. 
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MINISTRY OF COMMUNITY, FAMILY AND 
CHILDREN’S SERVICES 


3.01—Ontario Works 
Program 





BACKGROUND 


Under provisions of the Ontario Works Act, the Ministry of Community, Family and 
Children’s Services provides employment and temporary financial assistance to individuals 
on condition that they satisfy requirements intended to help them find and maintain paid 
employment. Financial assistance is provided through the Ontario Works program (Ontario 
Works) and is intended to be spent on basic living expenses for such things as food, clothing, 
personal needs items, and shelter. 


VFM Section 3.01 


Examples of Typical Monthly Benefit Payments 


Basic Allowance 





Maximum Shelter Allowance 





Source of data: Ministry of Community, Family and Children’s Services 


Additional assistance is available based on established need. Examples of items for which 
such assistance may be provided if circumstances warrant include: 


¢ health-related necessities, such as diabetic, surgical, and other medical supplies; 
¢ back-to-school and winter clothing allowances for eligible children; 
¢ basic dental and vision care; and 


* community and employment start-up benefits to assist in the cost of establishing a 
permanent residence or starting a new job. 





Ontario Works Program 29 


| VFM Section 3.01 | 








‘To be eligible for financial assistance, applicants must demonstrate a need for assistance by 
providing evidence that their liquid assets and income levels do not exceed specified 
amounts. Most individuals are also required to sign a participation agreement that requires 
them to take part in one or more of the following employment-assistance activities: 


¢ employment-support activities, which help participants become job ready through basic 
education, job-specific skills training, literacy training, or the Learning Earning and 
Parenting program for young parents to help them finish high school and prepare for 
the job market; 


* community participation, which provides unpaid placements in community-service 
activities designed to better the community and provide participants with practical work 
experience; and 


* employment placement and support to self-employment, which, respectively, place 
employment-ready participants in unsubsidized, competitive jobs or support them in 
pursuing self-employment opportunities. 


Since April 1, 1999, Ontario Works has been delivered on behalf of the Ministry by 47 
Consolidated Municipal Service Managers and First Nation delivery agents (collectively 
known as service managers). Funding for financial assistance to recipients through Ontario 
Works is cost shared between the province (80%) and the service managers (20%). 
Administration costs are shared equally by the province and service managers up to an 
annually established maximum for each service manager. 


For the 2001/02 fiscal year, the Ministry’s share of financial assistance benefits provided 
under Ontario Works was approximately $1.4 billion. The Ministry’s share of costs for 
program administration was $171 million. Ontario Works’ average monthly caseloads and 
consequent expenditures have steadily decreased in recent years as illustrated by the table 
below. 


Average Monthly Caseloads and Ministry Expenditures, 
1999/2000-—2001/02 









1999/2000 2000/01 2001/02 



















a Monthly Caseload Data ; 
Average monthly caseloads 262,436 215,614 189,321 
Number of dependents D1D.D47 253,908 222,945 
Total beneficiaries 577,983 469,522 412,266 


_ Annual Ministry Expenditures ($ million) 


Financial assistance for the year 











Administration costs for the year 





Source of data: Ministry of Community, Family and Children’s Services 
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The Ministry has also taken the initiative for, and has borne the cost of, an ambitious, 
province-wide revision of the information technology system and business processes to 
support the delivery of its social-assistance programs. This undertaking, called the Business 
‘Transformation Project, was, in part, to support the modernization of older computer 
systems and business processes at the limit of their usefulness into a new system that could 
deliver the new Ontario Works and the Ontario Disability Support programs efficiently and 
effectively. 


The new service-delivery system, including both a new information technology system and 
revised business processes, was implemented across the province by January 2002. This 
system was developed at a cost of approximately $400 million, as of March 2002, by the 
Ministry and Accenture (formerly known as Andersen Consulting), the arrangement for 
which was the subject of an audit and report by the Provincial Auditor in 1998; a Special 
Report to the Standing Committee on Public Accounts in 1999; and a follow-up report in 
our Special Report on Accountability and Value for Money (2000) on the Ministry’s 


implementation of recommendations from the Provincial Auditor’s 1998 Annual Report. 


AUDIT OBJECTIVES AND SCOPE 


Our audit objectives for Ontario Works were to assess whether the Ministry: 


¢ ensured that its established policies and procedures for the program were adequately 
adhered to by service managers to ensure that only eligible individuals received financial 
assistance and that the financial assistance provided was in the correct amount; and 


¢ monitored the effectiveness of the program in meeting its objectives, and, where 
necessary, took corrective action. 


The scope of our audit included a review and analysis of relevant ministry files, policies, and 
procedures as well as interviews with appropriate staff at the Ministry's head office and three 
regional offices. We also held discussions with staff of three Ontario Works intake-screening 
units, reviewed a sample of recipient files and held discussions with staff at the three service 
managers we visited, and obtained information from other service managers by means of a 
questionnaire. The intake units and service managers we visited together accounted for 
almost 40% of total program expenditures. 


Prior to the commencement of our audit work, we identified the audit criteria that we 
would use to conclude on our audit objectives. These were reviewed with and agreed to by 
senior management of the Ministry. 


Given the critical importance of the new service-delivery system to the current 
administration of Ontario Works and the fact that it was substantially completed and 
implemented across the province between May 2001 and January 2002, we also followed 
up on the recommendations we made in our 1998 Annual Report and those made by the 
Standing Committee on Public Accounts in 1999 regarding the administration of the 
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agreement between the Ministry and Accenture and assessed the adequacy of the new 
business processes and information technology system that were developed as a result of it. 


Our audit work was primarily conducted in the period from November 2001 to May 
2002, with emphasis on program expenditures and procedures during the 2000/01 and 
2001/02 fiscal years. We concentrated on areas with the largest program expenditures— 
basic needs and shelter assistance and community and employment start-up benefits. Our 
audit was performed in accordance with the standards for assurance engagements, 
encompassing value for money and compliance, established by the Canadian Institute of 
Chartered Accountants, and accordingly included such tests and other procedures as we 
considered necessary in the circumstances. 


In 1999, the Ministry’s Comprehensive Audit and Investigations Branch issued a quality 
assurance audit report on the implementation of the Consolidated Verification Process (a 
new process for determining recipient eligibility) in three municipal pilot sites. We reviewed 
this report but did not rely on the work performed since it did not address many of the 
issues covered in our report and predated the introduction of the Ministry’s new service- 
delivery system, which was implemented during 2001/02. 


OVERALL AUDIT CONCLUSIONS 


Both the Overall Audit Conclusions and the Detailed Audit Observations portions of this 
report are divided into two distinct sections: Part One deals with the administration of the 
agreement between the Ministry and Accenture; and Part Two deals with the administration 


of Ontario Works. 


Part One: Administration of the Agreement 


Although the new business processes and information technology system resulting from the 
Business ‘Transformation Project were rolled out to all service managers between May 2001 
and January 2002, at the time of our audit, most of the expected benefits to program 
delivery remained to be realized. For example: 


* Most service-manager staff we communicated with expressed considerable dissatisfaction 
with the Business Transformation Project, stating, for example, that the new service- 
delivery system was in many respects a step back from what had been available to them 
previously, that it had been inadequately tested, and that it was essentially still a work in 
progress rather than a finished product at the time of its release. 


¢ The new system did not provide service managers with accurate and reliable 
expenditure information for billing the Ministry for its share of the financial assistance 
provided to Ontario Works recipients. We were informed that prior to the introduction 
of the new system, such information had been readily available. 


With respect to the administration of the Business Transformation Project, we found the 
following: 


ee eeeeeeeeSSSS—SSSsSsSsSSSsSsS— 


26 2002 Annual Report of the Provincial Auditor of Ontario 





¢ The Ministry paid Accenture $246 million to March 2002, which was significantly 
more than the original $180-million payment cap. In our view, the basis for these 
payments continues to be questionable because: 


- savings attributed to the Business Transformation Project and hence to Accenture 
were exaggerated; and 


- contrary to the recommendations of the Provincial Auditor and the Standing 
Committee on Public Accounts to minimize out-of-cap expenditures, the Ministry 
paid Accenture $66 million outside the original $180-million payment cap. Of that 
amount, $22.2 million related to work that the Ministry had agreed to do in 
January 2001 and later but was unable to perform. 


e The new service-delivery system did not adequately support the administration of 
Ontario Works because of numerous unresolved systems defects. Business 
‘Transformation Project staff considered many of these defects to be emergency or high- 
priority items in need of repair. The deficiencies can generally be categorized as: 


- Ai failure to meet ministry or service-manager needs: The new information technology 
system often failed to provide needed information, provided it inaccurately, or 
provided it in a form that was not useful. Service-manager staff reported they found 
the system complicated to use and, for example, had great difficulty in determining 
why overpayments to recipients had been created and why they were recorded. 


- Unexplained errors or omissions: For example, in 2001, the information technology 
system inexplicably sent 7,110 discretionary benefit payments worth a total of 
$1.2 million to ineligible individuals. Also, at times the system did not permit 
caseworkers to issue cheques, which necessitated the issuing of manual cheques, and, 
at other times, it did not permit manual cheques to be recorded. 


- Internal control deficiencies: The information technology system exposed the 
program to an unnecessary risk of misappropriation of funds by allowing the same 
person to open new files, change banking information, make payments, and close 
the files without supervisory review or approval and with little chance of the changes 


being detected. 


Part Two: Administration of the Program 


With respect to the administration of Ontario Works, we concluded that the Ministry did 
not ensure that service managers adequately adhered to the Ministry’s policies and 
procedures for the administration of Ontario Works and, therefore, had little assurance that 
only eligible individuals received financial assistance in the correct amount. For example, we 


found the following: 


¢ Intake-screening units and the resultant two-step process for eligibility assessment were 
not meeting the anticipated objective of significantly reducing the number of lengthy 
in-office interviews for eligibility assessment. 
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¢ Ministry requirements for service managers to determine recipient eligibility for 
financial assistance and to provide that assistance in the correct amount were often not 
met. For example, for one of the service managers we visited, 95% of the files we 
reviewed lacked at least one of the information requirements necessary for establishing 
eligibility and the correct amount of assistance to be paid. 


¢ Other income reported by recipients of financial assistance that should have been 
deducted in whole or in part was often reflected incorrectly or not reflected at all in the 


benefits paid. 


¢ The reasonableness of payments for community and employment start-up assistance 
could often not be established, and, in some cases, the assistance was provided in excess 
of the maximum allowable amounts. 


¢ When spousal and child support was potentially obtainable from a former partner, we 
frequently found no evidence to indicate that it was being pursued. If such support is 
obtained, the amount of assistance paid by Ontario Works is reduced by the amount of 





that support. 


¢ The lack of accurate and reliable information about overpayments and inadequate 
computer system support for the collection function contributed to the inadequacy of 
service managers’ efforts to collect overpayments. 


We also concluded that the Ministry needed to improve its monitoring of the effectiveness 
of the program to determine whether it was meeting its objectives because: 


¢ Although it often cited decreases in the numbers of people receiving benefits as an 
indication of success, it had little information as to whether their departures were linked 
to positive program outcomes or had occurred for other reasons. 


¢ The circumstances under which specific termination codes indicating the reasons for 
recipients leaving the program were to be used were often not clearly communicated or 
well understood. Consequently, the codes were applied inconsistently by service 
managers and could not reliably be used to gather information on program outcomes. 


OTHER MATTER 


The Ministry reimburses service managers for 50% of administration costs up to an annual, 
negotiated amount. However, we found the Ministry had not assessed the cost-effectiveness 
of this funding, which was based on past funding amounts and not on actual caseloads. 
Ministry funding for administration costs ranged from a high of $1,596 per case to a low of 
$273 per case. 
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Overall Ministry Response 


The Business Transformation Project is one of the most ambitious government 
technology initiatives in Ontario to date. It took place in a very complex 
environment—in a Ministry that was charged with reforming its social 
assistance legislation and has a highly complex, multi-stakeholder delivery 
system and in a province that has significant geographic diversity. It designed 
a delivery system that took into account the diverse needs of social assistance 
recipients and the myriad of interrelated policies, business rules, procedures, 
and practices related to two dramatically changed programs: Ontario Works, 
delivered by municipalities, and the Ontario Disability Support Program, 
delivered by ministry offices. This business transformation was implemented 
through two levels of government that represent almost 300 delivery sites with 
7,500 users and serving more than 600,000 people. . 


The Business Transformation Project developed a new Service Delivery Model 
that features a Web-based system with on-line technology, a common province- 
wide database, telephone screening for eligibility, the use of third-party 
sources to verify eligibility, and an automated telephone system to provide 
recipients with up-to-date, easily accessed information on their personal case 
file. The Ministry believes that complex systems of this type normally require a 
substantial operating period in a live environment to deal with all the 
complexities that are inherent in the design of such large multi-user systems. 


A major principle for the Ministry was that benefits achieved through the 
Business Transformation Project would not only fully cover project costs, but 
would also continue to generate a return on investment. The Ministry believes 
that realization of this principle has been achieved. It is the Ministry’s view that 
savings attributed to the overall business transformation are substantial and 
substantial future savings will continue to accrue to taxpayers from this 
transformation. 


Many of the issues identified by the Provincial Auditor were known to the 
Ministry through its rigorous feedback process with users. We are continuing 
to address those issues with our delivery partners. The design allows for 
continuous improvement over the next few years, enabling the Ministry to 
make many modifications and enhancements, working hand in hand with our 
other partners—the Consolidated Municipal Service Managers. 


This project, then, has been about radical transformation of a large, complex, 
costly, and vital system. It has, at its foundation, been about improved services: 
taking the steps necessary to ensure that the law, the program, the 
administrative and management systems and processes, and the technology 
that together make up the social assistance system deliver the desired 
outcomes. It has been done in a way that has furthered the Ontario Public 
Service’s knowledge and understanding about how best to partner with others 
to make such radical change happen. 
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DETAILED AUDIT OBSERVATIONS 


During the mid-1990s, the computer systems used by the Ministry to process social- 
assistance payments had reached their practical capacity and were unable to support desired 
changes to improve program delivery and administrative effectiveness. The Ministry 
decided that it lacked the internal expertise and financial resources to develop the necessary 
business processes and technologies either in-house or with the assistance of outside 
contractors using traditional procurement methods. 


Instead, the Ministry chose to follow the common purpose procurement process, an 
innovative arrangement under which ministries and private-sector partners are to share the 
risk, investment, and future savings of jointly identifying and implementing new ways of 
delivering services. 


Thus, on January 27, 1997, the Ministry entered into a common purpose procurement 
agreement with Accenture (formerly known as Andersen Consulting) to undertake the 
Business Transformation Project. This Project entailed the development and implementation 
of new business processes and a province-wide information technology system operating in 
real time to allow service managers to deliver all social assistance services. 


At the time of our first audit of the agreement between Accenture and the Ministry in 
1998, we concluded that the Ministry had not demonstrated due regard for economy and 
efficiency in the contract terms agreed to or in the administration of the work performed to 
February 1998, and we made a number of recommendations in that regard. 


Although the Ministry’s ability to act on some of these recommendations was limited by the 
legally binding terms of the original agreement with Accenture, there were still significant 
opportunities to improve on the agreement’s administration and thus to address many of 
our 1998 recommendations. 


The need to act on the most significant of our concerns was reinforced by the Standing 
Committee on Public Accounts, which held hearings on our report in December 1999, 
and passed a motion that included the following recommendations. 


* The maximum-payment cap of $180 million included in the original agreement should 
not be increased and expenditures excluded from the cap should be minimized. 


* The Ministry should develop an auditable system of benchmarking to ensure the 
correct attribution of benefits to the Business Transformation Project. 


¢ The functionality of the technology originally contracted for should not be diminished. 
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Part One: ADMINISTRATION OF THE AGREEMENT 
BETWEEN THE MINISTRY AND ACCENTURE 


Payments in Excess of Payment Cap 


At the time of our original audit of this agreement in 1998 and its subsequent follow-up in 
2000, the Ministry expressed confidence that, despite the Provincial Auditor’s various 
concerns, ultimately, the payments to Accenture would be capped at $180 million, 
excluding certain specified items. However, the table below shows that payments to 
Accenture to March 2002 and the total budgeted expenditures at that time significantly 
exceeded the cap. 


Actual Payments and Budgeted Expenditures to Accenture, March 2002 


In-cap payments 
Out-of-cap payments 


Production support, help desk, and 
application maintenance 


Out-of-scope work 


Accenture performing ministry work 





* These payments to Accenture do not include amounts for hardware or third-party software 
purchased by the Ministry from sources other than Accenture, which totalled approximately 
$34.6 million at the time of our audit. We also noted that the total cost of the Ministry’s 
contribution to the Business Transformation Project, including the $34.6 million noted above, was 
$153.9 million to March 2002. 


Source of data: Ministry of Community, Family and Children’s Services 


The total cost of the Business Transformation Project to March 2002, including ministry 
expenditures and payments to Accenture, was therefore about $400 million. 


Our concerns with respect to each of the above types of out-of-cap payments are detailed 
below. 


PRODUCTION SUPPORT, HELP DESK, AND APPLICATION 
MAINTENANCE 

The items listed in the title above are the only items for which Accenture was to be paid 
outside the $180-million payment cap as outlined in the original 1997 agreement. 
However, as we noted in our 1998 audit report, the circumstances under which costs for 
some of these items could be incurred had not been clearly defined and the costs had not 
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been estimated, even though we thought then that they could be substantial. Costs for these 
items were subsequently budgeted at $19.7 million in May 2001; that budgeted amount 
had increased to $39.1 million by March 2002. 


Most of the actual expenditures for these items were incurred after the initial system release 
in May 2001. We examined a sample of items in this category and found that many were 
attributable to the new information technology system failing to function as intended, thus 
requiring critical design changes and increased technical support. We believe that many of 
the required changes should have been identified in pre-release testing and corrected under 
the warranty provisions of the agreement at no cost to the Ministry. In our view, the 
willingness of the Ministry to pay for these items outside the agreed-to payment cap appears 
questionable and unnecessary. 


Contrary to the recommendations made by both the Provincial Auditor and the Standing 
Committee on Public Accounts to minimize the number of items excluded from the 
maximum payment cap, the April 2000 amendment to the original contract provided for 
the following two additional items to be paid for outside the payment cap. 


OUT-OF-SCOPE WORK 


The new information technology system was essentially designed and developed in 2000 
based on the business requirements of the Ontario Works and Ontario Disability Support 
programs as established in July 1999. After July 1999, when new requirements for both 
programs needed to be incorporated into the new system design, doing so was considered 
out of scope and the work was billed for outside the payment cap, which we considered 
reasonable. 


However, we believe one part of this work should not have been designated as out of scope. 
The information technology system’s reporting function as originally designed and built did 
not meet many of the service-manager or ministry requirements. At the time of our audit, 
the Business ‘Transformation Project was developing and implementing changes to that 
function at an estimated out-of-cap cost of $2.3 million, $1.4 million of which had been 
paid as of March 2002. In our view, these costs ought to have been avoided by including 
the critical reporting requirements in the original information technology design. 


ACCENTURE PERFORMING MINISTRY WORK 


Although both the original agreement with Accenture and the subsequent amendment in 
April 2000 maintained the maximum payment cap at $180 million, neither the original 
agreement nor the subsequent amendment specifically defined the respective responsibilities 
of Accenture or ministry staff. Instead, some specific responsibilities were assigned to the 
Ministry in a January 2001 amendment to the agreement and subsequently, as specific task 
orders were approved. 


The Ministry was unable to complete some of that work; Accenture performed the work on 
behalf of the Ministry and was paid for it outside the cap, which had already been reached 
in early 2001. 
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Essentially, the Ministry paid Accenture $22.2 million to March 31, 2002 outside the 
payment cap for performing work for which responsibility had not been defined in either 
the original agreement or in the April 2000 amendment. This made the payment cap 
ineffective and demonstrates its failure as a protection against project cost overruns for the 


Ministry and, ultimately, for taxpayers. 


Cost and Benefit Pools 


For the Ministry and Accenture to share in the risk, investment, and future savings of the 


Business Transformation Project, in keeping with common purpose procurement principles, 


the agreement between them provided for payments to be made from a benefit pool to 
cover the accumulated amounts charged by each partner to a cost pool. (As noted in both 
our 1998 audit report and in our 2000 follow-up on that report, Accenture’s billing rates 
were substantially higher than the corresponding amounts charged by the Ministry.) Cost 
and benefit pool balances and the amounts actually paid as at January 2001 and March 
2002 were as detailed in the following tables. 


Cost Pool 


Accenture: 
In-cap 


Ministry: 


In-cap 
Out-of-cap | 





* Although costs to the Ministry cannot be divided into in-cap and out- 
of-cap costs, they are mainly related to in-cap activities. 


Source of data: Ministry of Community, Family 
and Children’s Services 
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Benefit Pool 


_ January 2001 March 2002 
million) —($ million) 


Reported benefits 274.6 5S 


Payments to Accenture Loa 246.0* 
Allocation to the Ministry 153.9 


*Of this amount, $180 million is related to in-cap activities. 





Source of data: Ministry of Community, Family 
and Children’s Services 


Both our 1998 Annual Report and our 2000 follow-up report described our two main 
reservations regarding the workings of the cost and benefit pools and hence the amount 
paid to Accenture, reservations we continue to hold: 


¢ First, it is still our view that the Ministry’s contribution to the cost pool is understated. 
For example, the total costs to the service managers for obtaining data extracts and 
setting up programs to process them to provide necessary management information are 
likely to be significant. However, those costs are not included in the cost pool even 
though they are a direct result of the information technology system’s reporting 
deficiencies. This topic is discussed again further on in our report (see Failure to Meet 
Ministry and Service-manager Needs). 


¢ Second, approximately 73% of the balance in the benefit pool, or $427 million as at 
March 31, 2002, had been attributed to the Consolidated Verification Process—a 
process for reviewing recipient eligibility for payments and the appropriateness of 
amounts paid. Contrary to our findings, the Ministry believes that the attribution of 
these benefits to Accenture is warranted in part because the Consolidated Verification 
Process introduced two major enhancements over the previous verification process, 
namely risk-ranking flags that automatically prioritize recipient files for review and third- 
party confirmation of information provided by recipients. As we discuss later on in this 
report, we found that neither of these features was being used as intended and therefore 
required improvements (see Eligibility Assessment Process Enhancements). 


In addition, many staff at the service managers we visited stated that the changes inherent in 
the Consolidated Verification Process were relatively minor in nature compared to previous 
processes and often did not affect the termination rates of recipients to any noticeable 


degree. 


Therefore, in our view, a significant portion of the reported benefits accumulated in the 
benefit pool continue to be highly questionable. 
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We also noted that since May 2001, most of Accenture’s costs for the Business 
‘Transformation Project have shifted to out-of-cap activities. In addition, a January 2001 
contract amendment provides for the extension of the two-year period for benefit accrual 
after the end of the contract “if, as, and when necessary in order to allow for Accenture to 
be fully reimbursed for all costs approved by the parties which are not subject to the cap.” 


Since the cost and benefit pool balances were reasonably similar at the time of this contract 
amendment, the Ministry has in effect abandoned the common purpose procurement 
principle of sharing risk and rewards by virtually guaranteeing Accenture full payment for 
all of its out-of-cap work, which is now substantial. 


Business Transformation Project Status 


In our 1998 report we noted that, at the time we completed our field work in February 
1998, the Business Transformation Project was significantly behind schedule in meeting the 
then-expected rollout date of June 1999. The Project was subsequently restructured, and, at 
the time of our follow-up work in 2000, the expected rollout of the new service-delivery 
system across the province was to occur in phases between February 2001 and January 
2002. 


This revised rollout schedule was essentially met since the system was rolled out to all 47 
service managers that administer Ontario Works between May 2001 and January 2002. 
However, to meet that schedule, a number of important features that were part of the 
original system design were either set aside or not satisfactorily completed. For example: 


¢ Monthly income reporting by recipients using a telephone-based, interactive-voice- 
response system, which was to have been an important part of improving administrative 
efficiency, had not yet been developed. 


¢ Data archiving of recipient information from the old computer systems so that it would 
be available in the new information technology system was included in the original 
system design but had not been implemented at the time of our audit in May 2002. 


¢ The new information technology system as rolled out lacked the reporting functions 
necessary to help ensure that Ontario Works is efficiently and effectively managed, as we 
discuss further on in the report (see Failure to Meet Ministry and Service-manager 
Needs). 


In addition, the system as rolled out suffered from a number of significant defects as 
discussed later in this report, many of which remained uncorrected at the time of our audit. 


Knowledge Transfer 


One of the objectives of the partnership between the Ministry and Accenture was to foster 
sufficient knowledge transfer from Accenture to ministry staff to allow the transition of the 
operation and maintenance of the new information technology system from Accenture to 
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the Ministry upon completion of the Business Transformation Project. To the extent that this 
objective was met, the Ministry would be self sufficient, thereby reducing its dependence on 
outside consultants. 


However, at the conclusion of the agreement term in January 2002, the anticipated 
knowledge transfer was not sufficiently advanced with the result that the Ministry was not in 
a position to operate and maintain the information technology system. Instead, Accenture 
and other private-sector consultants provided nearly all of the technical resources necessary 
for completing, maintaining, and operating the information technology system after January 
2002 at a substantial cost to the Ministry. For example, Accenture’s services were extended 
from January 26, 2002 to March 31, 2002 and then to May 31, 2002 to provide technical 
maintenance services at an estimated cost of $5.7 million. We understand that at the time of 
our audit the Ministry was in the process of finalizing a decision on a request for proposals 
for technical maintenance services after May 31, 2002 for Management Board of Cabinet 
approval. 


The New Service-delivery System 


The overall objective for revising the business processes and modernizing the supporting 
information technology system for the Ministry’s social-assistance programs was to provide 
service managers with the tools to enhance recipient services and improve the service- 
delivery system’s financial integrity while reducing the cost of program administration. More 
specifically, the Ontario Works service-delivery system was intended to: 


* reduce the time spent by caseworkers on clerical and other administrative duties thereby 
freeing up more time for providing services to Ontario Works recipients, including time 
spent on the program’s employment focus; 


* provide more timely and accurate determination of recipient eligibility thus reducing 
overpayments, inappropriate payments, and general system abuse; and 


* improve access to the information necessary for effective program management and 
ministry oversight of Ontario Works. 


However, service-manager staff we talked to and those who responded to our questionnaire 
expressed considerable dissatisfaction with the new service-delivery system. In many respects, 
service-manager staff indicated that the revised business processes and information 
technology system were no improvement at all and, in some cases, were a step back from 
what had previously been available to them. 


The Ministry recognized at an early stage of the Business Transformation Project the wealth 
of knowledge and expertise of municipal staff involved in the delivery of social services and 
considered ongoing consultation with service-manager staff vital to the Project’s ultimate 
success, particularly during the planning stages. Nevertheless, although many sessions were 
held to gather service-manager staff input into the design of the new service-delivery system, 
staff generally indicated that only input supporting the proposed design was listened to. 
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Service-manager staff informed us that decisions to reject recommendations they had made 
during the design phase often resulted in system deficiencies that later required correction. 
Some of the major problems reported by service-manager staff are listed below. 


¢ At the time of our audit, the new delivery system had not yet provided any noticeable 
efficiencies or time savings. In fact, numerous inefficiencies were created by the amount 
of manual work and number of “workarounds” required to deal with the information 
technology system’s many deficiencies. (A “workaround” is a procedure for coaxing a 
computer program into producing results that it has not been programmed to produce 
more directly.) In addition, service-manager staff advised us that they spent much extra 
time manually checking unreliable information produced by the information 
technology system. 


¢ Many service-manager staff we interviewed indicated that the information technology 
system had not been adequately tested and, in effect, was a work in progress at the time 
of its release. 


¢ Staff generally felt that the training they received on how to use the new information 
technology system was inadequate. Most staff identified specific areas where more 
training was still required. For example, according to service-manager staff, the training 
required for using the system’s reporting functions was scheduled to be offered in 
September 2002, almost one-and-a-half years after implementation began. 


Staff were also critical of the new intake-screening process because they found that it did not 
significantly reduce the number of in-office eligibility assessments held. 


SYSTEM INVESTIGATION REPORTS 


When service-manager staff encounter a problem with the information technology system, 
they initially contact their on-site business expert who reviews the problem and determines 
whether it can be readily resolved. If the problem cannot be resolved, it is reported to the 
Business Transformation Project’s help desk, which prepares and logs an issue ticket. These 
tickets are then referred to business analysts and technical experts for their consideration of 
the reported problem. 


If the problem is found to be valid, the analyst or technical expert creates a system 
investigation report (SIR). Because particular problems are likely to be reported by various 
local offices and ticketed a number of times by the help desk, duplicate tickets are 
consolidated into one SIR. 


As of February 2002, the Business Transformation Project had accumulated 10,600 SIRs. 
Of these: 


¢ nearly half were classified as “system defects” of which 540 were unresolved at the time 
of our audit; and 


* approximately 5,700 were considered emergency or high-priority items of which 
approximately 550 were unresolved. 
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Based on our review of the system and discussions with service-manager staff, identified 
system deficiencies can generally be categorized as: 


* a failure to meet ministry or service-manager needs; 
* unexplained errors and omissions; and 
* internal control deficiencies. 


Our observations concerning these system deficiencies are outlined below. 


FAILURE TO MEET MINISTRY AND SERVICE-MANAGER NEEDS 


Operational and Performance Information 


Access to adequate operational and performance information is critical to the efficient and 
effective operation of any program. However, in a number of instances we noted that, at the 
time of our audit, the new information technology system for Ontario Works either did not 
provide information that was needed or provided information that was unreliable. The 
problems described below were all ongoing at the time of our audit. 


Examples of information the new technology system could not provide included: 


* A listing of cumulative overpayments for each active recipient: This report would give 
both service managers and the Ministry the means to monitor the level and recovery of 
overpayments and help identify caseworkers in need of additional training. Although 
there was no plan to implement this type of report, the Business Transformation Project 
intended to introduce one that lists non-cumulative new overpayments by recipient in 
June 2002, which will be of limited use in our view. 


* A listing of overpayments that are not being pursued for recovery and that, in effect, are 
written off: The system allowed caseworkers to decide which overpayments not to 
pursue without supervisory approvals. As a result, supervisors need a listing of the 
overpayments not being pursued to identify and review the appropriateness of these 
decisions. At the time of our audit, there were no plans to implement such a listing. 


* A listing of recipients who lacked participation agreements or whose participation 
agreements were expired: We were advised that correction of this deficiency was 
planned for March 2002 but had been delayed as a result of labour disruptions. 


Examples of inaccurate or inadequate information provided by the new information 
technology system that had not been remedied at the time of our audit included the 
following: 


* In order to bill the province for its share of Ontario Works costs, service managers must 
determine the total amount paid to recipients. However, since the implementation of 
the new information technology system, the service managers have been unable to 
reconcile detailed listings of their payments to recipients with che summary expenditure 
reports produced for billing purposes. At the time of our audit, attempts to fix this 
problem were underway. As a result, service-manager billings to the province were 
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based on estimates, as detailed in a later section of this report (see Service-manager 
Claims for Financial Assistance Costs). 


¢ We compared the amount of overpayments recovered from inactive recipients in one 
service manager's own tracking system to amounts recorded as recovered by the new 
information technology system for the period from October 2001 to January 2002 and 
found that the totals were $38,300 and $309,000 respectively. Service-manager staff 
were unable to explain the discrepancy. 


¢ According to service-manager staff, monthly reports of activities and outcomes for the 
Consolidated Verification Process generated by the new technology system were 
inaccurate because they contained duplicate recipients and recipients included in error. 
Because of these inaccuracies, the reports were not useful. 


e The information technology system produces an exception report that identifies 
instances of caseworkers overriding a system-generated value. However, this report is 
simply an undifferentiated list of overrides, many of which are due to workarounds and 
simple errors being corrected. Although the list includes overrides that resulted in 
changes to amounts paid, those overrides cannot be isolated from other items and 
therefore the information is not useful. 


Some service managers request data extracts from the Ministry so that they can process and 
analyze the data themselves to obtain information that is not otherwise readily available as 
well as to compensate for some of the deficiencies noted above. Data extracts have been 
provided but analyzing the data involves costs to service managers for setting up the 
necessary computer programs. In addition, we were informed by one of the service 
managers that had used this approach that the data provided was not completely reliable 
and accurate. For example, its monthly data extract continued to include approximately 
600 cases that had previously been transferred out of the service manager’s jurisdiction. 


System Complexity 


An information technology system should be reasonably easy to use and its users should be 
provided with sufficient training to obtain the required working knowledge of the new 
system. However, service-manager staff informed us they found the new technology 
complicated to use and that, combined with a lack of training, has led to errors, extra work, 
and stress for users: 


¢ For example, according to the service managers, determining why an overpayment was 
created and recorded on the system is difficult and time consuming. Service-manager 
staff informed us that the previous computer system allowed caseworkers to review their 
actions in order to help check the accuracy of the results. The new information 
technology system does not allow such reviews. In addition, caseworkers were sometimes 
unaware of instances when they had created overpayments. Although a new feature was 
added to prompt caseworkers when an overpayment was about to be created, after a 
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subsequent update to the information technology system, that prompt appeared only 
intermittently. 


¢ One service manager hired three “troubleshooters” to review and help establish the 
accuracy of payments to recipients. The service manager informed us that in one month 
alone the troubleshooters investigated 148 cases identified as likely to be affected by 
errors and found that: 


- $123,457 in benefits that recipients were not entitled to had been issued; 


- $106,687 was paid correctly but had been recorded on the system as overpayments; 
and 


- $7,743 that recipients were entitled to had not been paid. 


These results are troubling because they suggest that the information technology system is 
not functioning properly or that staff are not using the system properly or some 
combination of both. 


UNEXPLAINED SYSTEM ERRORS 


The information technology system was not operating as consistently or reliably as expected. 
A number of errors were occurring for reasons that service managers could not explain. We 
noted that many of these errors were caught due to the vigilance of caseworkers. However, 
the errors made by the information technology system so far do not inspire confidence and 
raise the possibility that other significant problems may go undetected. 


Below are examples of unexplained information technology system errors, most of which 
had not been corrected at the time of our audit: 


* Between May and December of 2001, the information technology system inexplicably 
produced 7,110 discretionary benefit payments totalling $1.2 million to recipients who 
were ineligible for Ontario Works assistance of any kind. 


* In some instances, when a caseworker attempted to cancel a cheque, the information 
technology system reissued the cheque the next day, The cancelling and reissuing of 
these cheques continued until the Business Transformation Project office effectively 
cancelled them. 


* When a caseworker cancels a cheque, any related overpayment amount should be 
adjusted accordingly. We were informed, however, that this did not always occur, which 
resulted in overpayment balances being overstated. 


* We were informed that amounts on cheques did not always agree with the amounts on 
the attached cheque stubs or with corresponding amounts on the paylist. We 
understand that this was corrected subsequent to our audit. 


* In some instances, caseworkers requested that a cheque be sert to a recipient only to 
have the information technology system fail to comply. Service-manager staff then had 
to prepare a manual cheque. 
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In many cases, money received from the Family Responsibility Office to reimburse the 
service managers for prior payments made to spouses could not be recorded as intended 
because doing so would trigger the information technology system to send another 
cheque for the reimbursement amount to the recipient. 


Many manual cheques issued to recipients and third parties on their behalf could not be 
recorded. For example, for one month alone, one service manager could not record on 
the system 41 manual cheques totalling $72,842. 


The information technology system automatically produced tax slips for all recipients 
who received Ontario Works assistance in 2001 so that recipients could include the 
amounts on their tax returns. However, a number of these slips reported unreasonable 
amounts, including amounts for —$1.2 million and —$194,000. Although the 
Ministry informed us that none of the tax slips cited had been mailed out, we found no 
evidence to indicate that corrected tax slips had been subsequently issued. In addition, 
the inability of service managers to record some reimbursements and manual cheques 
on the system also resulted in recipients receiving inaccurate tax slips. 


The system automatically prints form letters to recipients relating to a number of issues 
such as overpayments. According to the service managers, the information in many 
letters is inaccurate or of no value. For example, letters have been printed informing 
recipients that they have received a $0 overpayment and that they are “obligated to pay 
this debt.” As a result, service managers have had to visually review all letters produced 
by the information technology system to ensure that they are accurate and useful. This 
has increased the amount of time spent on administrative functions. 


INTERNAL CONTROL DEFICIENCIES 


Information technology systems generally include a number of internal controls to help 
ensure that intentional or unintentional errors do not occur. However, we noted that the 
new information technology system lacked certain basic internal controls, including the 


following: 


The restriction of certain tasks to specific individuals helps ensure that they cannot 
perform combinations of tasks on their own that could result in errors or the 
misappropriation of funds. However, we noted the system lacked such controls over the 
payment function. For example, the system allowed the same person to open a new file 
or reactivate a terminated one, change banking information, make payments to 
recipients, and close the file without supervisory review or approval. The likelihood of 
such payments being detected was low because the system did not produce reports on 
cases opened or closed by caseworkers. This situation exposed Ontario Works to an 
unnecessary risk of misappropriation of funds. 


The system was supposed to produce an audit trail that allowed service managers to 
track any changes made to files by user identification. However, we noted that an audit 
trail was not always maintained, as there have been instances where individuals who 
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made no changes to a file have overridden a previous user identification. In some 
instances, users with read-only access have also overridden previous user identifications. 


* Since the information technology system that supports both Ontario Works and the 


Ontario Disability Support Program is integrated and province-wide, we would have 
expected it to be programmed to prevent recipients from receiving both Ontario Works 
and Ontario Disability Support payments for the same time period and to prevent 
duplicate or multiple Ontario Works payments being sent to the same individual by 
different service managers. However, service managers informed us that they had found 
instances of these kinds of duplicate and multiple payments. 


* The read-only access function does not permit users to view all information. To gain 


viewing access to all information, users must be given full edit rights, which is 
undesirable. 


¢ Service managers informed us of instances when individuals, including those with read- 


only access, have recorded an overpayment or issued a cheque without being aware that 
such an operation had occurred. 


* Many service managers also noted that the workarounds they must use to make the 


information technology system perform better could make it difficult to pursue 
instances of suspected fraud in the future. The integrity of the system and the data are at 
risk in instances where false information is entered into the system during the course of a 
workaround. One service manager estimated that prior to the new system's 
implementation it was referring for prosecution approximately four cases of suspected 
fraud a month, but, almost one year after system implementation, it was not referring 
any cases for prosecution. The service manager attributed this decrease to a lack of 
integrity of information in the system as a result of workarounds and the way 
information was organized. 





Conclusions on the Administration of the Agreement Between the 
Ministry and Accenture 


in light of the observations detailed in this report, it is our view that the 
common purpose procurement principles of sharing risk, investment, and 
rewards were not provided for in the agreement between the Ministry and 
Accenture or in its subsequent amendments and were clearly not adhered to. 
In effect, the Ministry accepted most, if not all, of the risk for the Business 
Transformation Project while Accenture received a disproportionate amount of 
the rewards. 





If the Ministry again decides to undertake a common purpose procurement 
project of this complexity and magnitude, it needs to ensure that the project is 
delivered on time, on budget, and meets the business needs of the Ministry and 
its service-delivery agents. Based on the Ministry’s experience in this case, we 
believe that the potential benefits of common purpose procurement projects 





do not justify the risk involved. 
L 
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Ministry Comment a 


It is the Ministry’s view that the common purpose procurement form of 
contracting was well suited to this project because it allowed the parties to 
establish the overall deliverables as part of the Master Agreement. Over the 
term of the contract, the parties then evolved the business and technology 
solution with the participation of its many stakeholders. The Task Orders 
outlining this work detailed the respective responsibilities, the resources, 
timing, and specific outcomes that formed part of the contract. Both parties 
assumed a significant investment and agreed to share the risk and rewards 
proportional to their investment. The terms of the Master Agreement and 
subsequent contractual documents were strictly adhered to as validated by 
Contract Compliance Reviews performed during the project and at project 
closure. 








The Ministry agrees that all common purpose procurement contracts should 
ensure that the projects are delivered on time and on budget and meet the 
business needs of the users and believes that those criteria have been met. The 
benefits of this form of contracting should always be balanced against the 
risks involved in delivering these large, complex projects. This project 
furthered the Ontario Public Service’s knowledge and understanding about 
how best to partner with others to make such radical change happen. 
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Part Two: ONTARIO WORKS PROGRAM 
ADMINISTRATION 


In preparation for the introduction of the new service-delivery system for Ontario Works, 
the Ministry fully implemented a new, two-step application process for eligibility assessment 
in January 2001. Since that time, individuals applying for assistance are to call one of seven 
intake-screening units and provide basic personal information about such matters as their 
income, assets, and current accommodation. Operators enter this information into the new 
technology system and, based on the information provided, advise the applicant whether he 
or she is likely to be eligible for assistance. 


Regardless of the opinion expressed by screening-unit staff at this stage, individuals who 
choose to continue with the application process are referred to a service manager and, in 
most cases, are scheduled for an Ontario Works information session and a subsequent 
personal interview. During the personal interview, the applicant must produce key 
documents to establish eligibility for financial assistance. In addition, the service manager is 
required to confirm some of the information provided during the interview with third 
parties such as a credit agency or other benefit providers. 


In cases where financial eligibility is determined, the applicant is also required to enter into a 
participation agreement, which entails a commitment to participate in employment-related 
activities designed to help the participant find and keep a job. The participant’s continued 
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eligibility for financial assistance is periodically reassessed at frequencies that vary with the 
perceived risk of ineligibility. 


In cases where the application for financial assistance is denied, applicants can first ask for an 
internal review of the decision by the service manager, after which the decision may be 
appealed to the Ministry’s Social Benefits Tribunal. 


Intake-screening Units 


The two-step process for eligibility assessment was designed to increase program efficiency 
and eftectiveness principally by: 


* permitting applicants to obtain preliminary eligibility assessments at an earlier stage of 
the application process through a convenient telephone call that lasts, on average, only 
15 to 20 minutes; 


* reducing the number of in-office interviews for applicants who will ultimately be found 
ineligible for financial assistance, interviews that typically take up to two hours; and 


* enhancing the consistency and effectiveness of the eligibility assessment process. 


However, we found that the new eligibility-assessment process was not meeting the objective 
of significantly reducing time-consuming, in-office interviews. 


Ministry documentation provided to us indicated that half of all applicants were expected to 
be screened out as a result of their initial telephone inquiries to the intake-screening units. 
However, records for 2001 indicated that the results obtained by screening units fell far 
short of meeting this goal. 


* Based on a four-month ministry study, 40% of all applicants bypassed the screening 
units altogether and applied for assistance directly to a service manager. 


* Eighty-seven percent of all callers to screening units in 2001 chose to proceed to a 
personal, in-office interview at a service manager, regardless of the preliminary 
assessment provided by the screening unit. 


We also found that the Ministry did not have information that could demonstrate the 
effectiveness of the two-step eligibility-assessment process. For example, it did not have any 
information with respect to: 


* the number of applicants who were advised by screening unit staff that they were likely 
to be ineligible for financial assistance but proceeded to a personal interview and were 
found to be eligible; or 


* the number of applicants who were advised by screening unit staff that they were likely 
to be eligible for financial assistance, but were found to be ineligible at a later stage. 


Service-manager staff we talked to expressed the following additional concerns: 


*  Screening-unit staff who answer applicants’ calls were in some cases not as familiar with 
Ontario Works’ requirements as they should have been. 
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¢ The Ministry’s standard of answering 80% of all telephone calls within 60 seconds was 
often not met. For example, information for the first three months of 2002 for one of 
the screening units we contacted indicated that calls were answered within one minute 


only 14% to 72% of the time. 


Recommendation 





The Ministry should obtain the information necessary for assessing the 
efficiency and effectiveness of its eligibility-assessment process for the 
Ontario Works program, determine whether the intake-screening units are 
meeting expectations, and, where necessary, take corrective action. 








Ministry Response 





The Ministry agrees. Based on experience with early implementation, the 
Ministry is working closely with the Ontario Works service managers to 
enhance performance indicators, which will assist in monitoring effectiveness 
and efficiencies generally and specifically with regard to the intake-screening 
units. . 
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Recipient Eligibility 

Applicants must provide the service manager with the information necessary to demonstrate 
their eligibility for financial assistance and to determine the correct amount of assistance to 
be paid. Ministry policy requires that at the time of the initial eligibility assessment and for 
subsequent eligibility reassessments copies of certain documents must be placed on file and 
the visual verification of other information must be noted on file. Document copies that 
must be kept on file include birth certificates, legal documents relating to marital status and 
support agreements, and tax assessments. Information that must be visually verified and 
noted on file includes social insurance and health card numbers, banking information, and 
school records for dependent children. Any single piece of missing information could have a 
significant impact on the eligibility assessment and the determination of the correct amount 
of assistance to be provided. 


Our examination of a sample of recipient files found that the Ministry’s requirements for 
obtaining and verifying all of the information necessary to establish eligibility and amounts 
to be paid were often not met: 


¢ On average, approximately three-quarters of the recipient files for which initial benefits 
had been granted lacked at least one of the information requirements and in some cases 
up to three or four. For one service manager we visited, we noted that 95% of the files 
we reviewed lacked at least one of the information requirements. 
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* On average, approximately three-quarters of the recipient files for which a subsequent 
eligibility reassessment had been undertaken continued to lack at least one of the 
required items of information. 


We noted that the rates at which required information was lacking were comparable to 
those cited in our 1996 report on the Ministry’s Provincial Allowances and Benefits 
Program. Therefore, little if any improvement has been realized in this area since that time. 
We also noted that the Ministry’s own compliance-review process for recipient files found 
many of the same information deficiencies that we found, and no corrective action had 
been taken. 





Recommendation 


To ensure that all recipients are eligible to receive Ontario Works financial 
assistance and that the assistance provided is in the correct amount, the 
Ministry should reinforce with service managers its requirements for obtaining, 
documenting, and correctly assessing the required recipient information. 


Ministry Response 


The Ministry places a high priority on ensuring that only eligible people receive 
the accurate amount of financial assistance and will reinforce this with Ontario 
Works service managers through its compliance review and monitoring 
process. 
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Eligibility-assessment Process Enhancements 


With the introduction of the new service-delivery system in 2001/02, the Ministry 
introduced two enhancements to the eligibility-assessment process for recipients—automatic 
risk flagging of files and third-party confirmation of information provided by recipients. 
However, neither of these were consistently used and, therefore, did not meet ministry 
expectations. 


FLAGGING HIGH-RISK CASES FOR ELIGIBILITY REVIEW 

The new information technology system is programmed to assign risk flags to active 
individual cases in the system based on a variety of assessed characteristics. All cases are 
assessed as high, medium, or low risk for potential non-compliance with program eligibility 
requirements. 


Cases are assessed as high risk if they meet any one of a number of characteristics, such as 
having declared accommodation costs exceeding 80% of the total assistance provided or not 
having had an eligibility review conducted within the previous eleven months. Service 
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managers are expected to conduct reviews of cases flagged as high risk on a priority basis and 
before reviewing cases assessed as medium or low risk. 


However, we found that the information technology system’s risk-flagging feature was not 


effective in flagging the highest risk cases and was not effectively used to prioritize case 


reviews for the following reasons: 





Recommendation 


To better identify and rank the highest risk cases for review and to ensure that 
those reviews are conducted on a priority basis, the Ministry should: 


At the time of our audit, large numbers of cases were flagged as high risk and therefore 
in need of review. For example, 10% of all cases for one service manager were flagged as 
high risk, which represents about twice as many cases as can be reviewed in a month. 
However, service managers were unable to prioritize their workload within this category 
to determine which cases should be reviewed first because: 


- cases that had not been reviewed for eleven months were not differentiated from 
those that had not been reviewed for two or more years; and 


- acase that met one of the criteria for high risk could not be differentiated from 
those that met three or four criteria. 


Service managers could not alter the risk-assessment criteria to reflect local conditions. 
Por example, one service manager had a large proportion of cases flagged as high risk 
because 80% or more of the assistance amount was spent on accommodation. However, 
even after these cases were reviewed and recipients were found to be eligible, the cases 


continued to be flagged as high risk. 
Many of the cases flagged as high risk had not been reviewed for prolonged periods of 


time. For two of the service managers we visited, we found high-risk cases that had not 
been reviewed for between two and three years. 


Two service managers we visited were conducting reviews for cases rated as medium and 
low risk while other cases rated high risk were not reviewed. For example, of the cases 
assigned for review during a one-month period at one service manager, 24% were rated 
high risk, 44% were rated medium risk, and 32% were rated low risk while cases 
identified as high risk remained unreviewed. 


consider refining the criteria used to identify and rank cases in need of a 
review; and 

ensure that service managers prioritize reviews on the basis of assessed 
risk. 
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Ministry Response 








The Ministry agrees. The Ministry is currently reviewing its risk criteria and 
monitoring process in conjunction with Ontario Works service managers to 
pernanes this process. 

















THIRD-PARTY CONFIRMATION OF INFORMATION PROVIDED BY 
RECIPIENTS 


In our 1996 report on the Provincial Allowances and Benefits Program, we recommended 
that the Ministry enter into information-sharing agreements with other benefit providers 
and jurisdictions to help ensure that information provided by recipients is correct and that 
errors or omissions resulting in inappropriate eligibility determinations are prevented or 
detected. 


As part of the introduction of the new service-delivery system, the Ministry acted on that 
recommendation and entered into a number of third-party information-sharing 
agreements. Confirming the completeness and accuracy of information provided to service 
managers with third parties is now mandatory in some instances and left to the discretion of 
caseworkers in others. Mandatory third-party confirmations at the time of both the initial 
eligibility determination and for subsequent reviews include: 


* confirming information with respect to Employment Insurance benefits provided by 
the federal government; and 


e confirming personal credit information with a major credit-rating agency. 


Mandatory third-party confirmations of information provided only at the time of 
subsequent reviews include: 


* confirming support payments with the Family Responsibility Office when support is a 
potential issue; and 


* confirming income information with the Canada Customs and Revenue Agency. 


However, we found that for almost half of the sample of files we reviewed that should have 
included mandatory third-party confirmations, at least one such confirmation had not been 
undertaken. For one service manager, we found that 70% of such files were missing at least 
one of the required third-party confirmation. In addition, we found that discretionary 
third-party confirmations, for example, confirming spousal support with the Family 
Responsibility Office at the time of the initial eligibility assessment, were seldom done, even 
though in our opinion they often should have been carried out. 









Recommendation 


To help ensure that information provided by recipients of Ontario Works 
assistance is complete and accurate and that errors or omissions resulting in 
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inappropriate eligibility determinations are detected and prevented, the 
Ministry should: 


e assess the advisability of making all mandatory third-party confirmations at 
the time of a subsequent eligibility review also mandatory at the time of the 
initial eligibility assessment; and 


e reinforce with service managers its requirement that all mandatory third- 
party confirmations be conducted as required. 





Ministry Response 


The Ministry agrees. The Ministry will assess the advisability of expanding 
mandatory third-party confirmations and reinforce with Ontario Works service 





managers the requirement for third-party confirmation. 


Reporting of Other Income by Assistance 
Recipients 


Ministry policy requires that all recipients of Ontario Works financial assistance report to 
their service manager any other income they receive in a month. The new information 
technology system was to automate this reporting requirement using a telephone-based 
interactive-voice-response system. However, this aspect of the new technology was not 
developed, and recipients continued to be required to submit a monthly income-reporting 
statement disclosing non-assistance income; that information is then manually entered into 
the system. 


If, in any of the first three months of receiving financial assistance, a recipient reports 
employment earnings, in the following month the amount of assistance is reduced by the 
amount of the reported earnings. After three months, recipients may recetve employment 
earnings up to a base amount, ranging from $143 to over $400 per month depending on 
the makeup of the family unit, without affecting their financial-assistance entitlements. 
Reported employment income in excess of the base amount is clawed back at the rate of 
75% during the first year, 85% during the second year, and 100% thereafter. Income from 
sources other than employment is deducted in full. 


We reviewed a sample of files for which income-reporting statements had been received or 
for which monthly benefit entitlements had been reduced due to reported income and 
found discrepancies in approximately one-quarter of the files. These discrepancies included: 


* reported income that was incorrectly reflected in the following month’s assistance 
payment or was not reflected at all; and 


° instances where assistance payments were reduced but either the reported income did 
not substantiate the reduction or the relevant income-reporting statement could not be 


found. 
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We also noted that the Ministry’s policy is not clear as to whether an income-reporting 
statement is to be submitted if no income is earned in a month. Two of the service managers 
we visited required an income-reporting statement to be submitted every month whether or 
not the recipient had any income while the other service manager did not require reporting 
from recipients who had no income. 


Recommendation 


To help ensure that financial assistance provided by the Ontario Works 
program is in the correct amount, the Ministry should reinforce the 
requirement that service managers correctly reflect other reported income in 
the financial assistance provided. 


Also, the Ministry should clarify whether or not monthly income-reporting 
statements are required from assistance recipients who have no income to 
report in a given month. 


Ministry Response 


One of the cornerstones of the Ministry’s reform of the social services delivery 
system is ensuring that participants are issued the correct amount of financial 
assistance. The Ministry will emphasize with Ontario Works service managers 

the requirement that all reported income is to be correctly reflected in financial 
| assistance provided. 








| 





Community and Employment Start-up Assistance 


In addition to financial assistance for basic needs and shelter, Ontario Works recipients may 
also be eligible to receive community or employment start-up assistance. Community start- 
up assistance may be granted for such items as moving expenses and the last month’s rent for 
new accommodation. Employment start-up assistance may be provided for such items as 
appropriate work wear or transportation costs for employment or employment-assistance 
activity. In the 2000/01 fiscal year, approximately $36.8 million and $11.6 million was 
provided for community and employment start-up assistance respectively. The table below 
shows the maximum benefit amounts that may be provided in any 12-month period. 
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Community Start-up and Employment Start-up 
Assistance Amounts, 2001/02 





Community start-up 
assistance 





Employment start-up 
assistance 





Source of data: Ministry of Community, Family and Children’s 
Services 


A ministry directive defines the specific circumstances under which a recipient is eligible for 
community and employment start-up assistance and requires that those circumstances be 
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documented. However, the directive does not define eligible expenditures. Instead it 
provides examples of the type of expenditures that may be reimbursed under either 
program. In addition, the directive does not require a detailed listing of the costs incurred or 
an assessment of their reasonableness. We also noted that although receipts are required to 
support employment start-up assistance payments, there is no such requirement for 
community start-up assistance payments. 


We found that local practices for obtaining supporting documentation for start-up 
assistance provided under the two programs varied widely among the three service 
managers we visited. One service manager required receipts to be filed for employment 
start-up purchases only; another required receipts to be filed for community start-up 
purchases only; and the third required a list of items to be purchased and their estimated 
costs for community start-up only. 


We reviewed a sample of files for which either community or employment start-up 
assistance had been provided and noted the following: 


¢ Contrary to the Ministry’s requirement, there was often no evidence on file that the 
community or employment start-up event for which assistance had been provided had 
occurred. 


¢ In the absence of receipts for the items acquired, it is not possible to assess the 
reasonableness of the amounts paid. 


¢ A number of assistance payments were for ineligible items or were otherwise 
questionable. For example, community start-up assistance was provided for such things 
as rent arrears, utility bills, and mortgage payments, in some cases to the maximum 
amount of $1,500. 
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We also noted instances where assistance was provided in excess of the established maximum 
amounts. For example, annual maximum amounts for both community and employment 
start-up assistance were often provided more than once in a 12-month period and, in one 
case, twice in the same month. 





Recommendation 


To help ensure that community and employment start-up assistance provided 
under the Ontario Works program is reasonable in the circumstances, the 
Ministry should: 


¢ reinforce with service managers its requirement to document and provide 
community and employment start-up assistance only in eligible 
circumstances and not in excess of the maximum amounts; and 

° require service managers to obtain a list of items to be reimbursed, assess 
the reasonableness of the amounts of assistance requested, and obtain 
receipts to substantiate all actual costs incurred. 


Ministry Response 


The Ministry agrees. The Ministry will reiterate with Ontario Works service 
managers the requirements to assess the reasonableness and accuracy of 
community and employment start-up assistance. 











Pursuing Spousal and Child Support 


Under provisions of the Ontario Works Act, recipients of financial assistance are required to 
make all reasonable efforts to pursue any spousal and/or child support they may be entitled 
to. If the caseworker is not satisfied that the recipient is making reasonable efforts in that 
regard, the caseworker may determine that the person is not eligible for basic financial 
assistance or reduce the amount of assistance by the amount of support that, in the 
caseworker’s opinion, would be available ifa reasonable effort had been made to obtain it. 
In addition, the recipient may be referred to a Family Support Worker who may assist the 
recipient in pursuing spousal and/or child support. 


In the case of court-ordered support, rights to the support payments may be assigned to the 
service manager, which is usually done when support is not being regularly paid. In these 
cases, the recipient receives full Ontario Works assistance entitlements, and it is the service 
manager's obligation to collect the court-ordered support. 


The Ministry does allow for circumstances where it is not reasonable to pursue support. For 
example, a caseworker may waive the obligation to pursue support when the recipient’s 
former partner cannot be located, is in a penal institution, or is likely to become violent if 
support ts pursued. Waivers for pursuing support are generally valid for a three-month 
period, after which the obligation to pursue support must again be assessed. 
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The degree of review a caseworker performs when assessing the adequacy of recipients’ 
efforts to pursue support is a function of the caseworker’s professional judgment. This results 
in inconsistent practices in this area and ultimately could lead to a lack of assurance that all 
reasonable efforts to obtain support have been taken. 


We reviewed a sample of files for which spousal or child support was potentially obtainable 
and, in most cases, found no evidence to indicate that support was in fact being pursued. 
Even in those cases where the recipient had been referred to a Family Support Worker, we 
still often found no evidence to indicate what steps, if any, had been taken to pursue 
support. 


We also found that for most cases where a waiver to pursue support was on file, the waiver 
had expired. In addition, we often found no evidence on file to indicate that the caseworker 
had received, verified, or assessed any information to determine whether the waiver to 
pursue support was warranted either initially or periodically afterward. 





Recommendation 


To help ensure that Ontario Works recipients who may be eligible for spousal 
and/or child support actively pursue such support, the Ministry should ensure 
that service managers: 


* ascertain and are able to demonstrate that all recipients entitled to such 
support have taken reasonable efforts to attain it; and 

¢ adequately document the information received, assessed, and verified in 
issuing a waiver to pursue support, and document the reassessment of the 
decision at the time the waiver expires. 





Ministry Response 


The Ministry agrees. The Ministry will reinforce with Ontario Works service 








managers the current file documentation requirements for family support. 


Recipient Overpayments 


Overpayments occur when the amount of assistance provided to recipients exceeds the 
amount they are entitled to receive. The amounts of unrecovered overpayments to 
recipients are substantial. As of February 2002, Ministry records indicated that 
overpayments outstanding on over 49,000 active accounts totalled $77.4 million. 
Overpayments outstanding for approximately 274,000 inactive accounts totalled 

$336.9 million as of that date. However, we understand that the reliability of this summary 
information is somewhat questionable because many of the individual account balances, 
which were the basis of the summary information, were found to be in error when 


reviewed. 
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Overpayments to active recipients are generally recovered through a 5% reduction to the 
recipient s monthly assistance payments until the overpayment is recovered in full. 
Overpayments to inactive clients are to be recovered through the collection efforts of service 
managers. Although overpayment recoveries are to be shared between the Ministry and the 
service manager on an 80:20 basis (the same basis on which they were originally funded), 
the Ministry has not assessed overpayment-recovery procedures for service managers in the 
last four years. 


We found that service managers’ efforts in collecting overpayments from inactive accounts 
were not adequate and were adversely affected by the following factors: 


* Staff resources assigned to the collection function were often inadequate. For example, 
at one service manager we visited, one person was assigned to recover overpayments 
from 10,000 accounts representing $10 million in outstanding receivables. 


* One service manager had made no attempt to collect outstanding overpayments after 
the introduction of the new information technology system in January 2002 because of 
the difficulty in determining the reasons for overpayments and in correcting 
outstanding balances for individual accounts after that date. 


* Since the information technology system does not support the collection function and 
cannot interface with the service manager’s own computer collection systems, the 
collection process is labour intensive and prone to manual input errors. 


* Service managers cannot effectively age or readily identify many old outstanding 
balances that are likely uncollectible and therefore should be written off. 


In light of the above-noted concerns, actual recoveries of overpayments from inactive 
accounts were minimal for the service managers we visited. For example, total recoveries 
averaged approximately 2% of the average amounts outstanding during 2001. 





Recommendation 


To maximize the recovery of overpayments to inactive recipients of Ontario 
Works assistance, the Ministry should: 


* ensure that its information technology system correctly indicates 
overpayment balances, allows the reasons for overpayments to be readily 
determined, and better supports the collection function; 


* ensure that service managers actively pursue the recovery of 
overpayments from inactive clients where warranted; and 


° consider the development of a policy for writing off uncollectible accounts 
so that uncollectible outstanding accounts can be identified and written off 


on a timely basis. | 
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Ministry Response 





The Ministry agrees. The Ministry is committed to improving the capacity of the 
information technology system to support the collection and recovery of 
overpayments by Ontario Works service managers. 





The Ministry will consider the development of a policy for writing off 
uncollectable accounts so that the outstanding uncollectable accounts can be 
identified and written off on a timely basis. 


Service-manager Claims 
for Financial Assistance Costs 


Each service manager is required to submit a monthly claim to the Ministry for its 80% 


share of actual financial-assistance payments provided to Ontario Works recipients. Prior to 
the introduction of the new information technology system, we understand that service 
managers had accurate and reliable expenditure information on the financial assistance they 
provided, which was used to prepare their monthly claims to the Ministry. However, since 
the introduction of the new information technology system, accurate and reliable 
expenditure information is no longer available. Although the new system produces both a 
summary expenditure report for Ontario Works benefits as well as a detailed payment 
listing, information with respect to total payments contained in these reports differed 


significantly and both differed again from the service managers’ banking records. 


As a result, two of the three service managers we visited could no longer submit monthly 
claims to the Ministry based on their actual Ontario Works program expenditures. Instead, 
the Ministry funded these service managers based on an average of three months’ actual 
expenditures prior to the introduction of the new system, and, in some cases, with 
adjustments that the service managers did not understand and could not explain. 


The third service manager had the necessary resources and expertise to process and analyze 
ministry-provided data extracts to produce program-expenditure reports that could be 
reconciled with its banking records. It therefore was able to continue to submit monthly 
claims to the Ministry based on its actual program expenditures after the introduction of the 
new system. We noted that the unexplained differences between this service manager's 
actual monthly expenditures and the summary expenditure reports produced by the system 
were substantial. For the six months prior to April 2002, these differences averaged 

$1.2 million per month and were as high as $5.4 million out of total expenditures of 
approximately $42 million. 








Recommendation 


To enable service managers to submit monthly claims to the Ministry for their 
share of actual Ontario Works assistance benefits provided, the Ministry 











Ontario Works Program a 


VFM Section 3.01 


| VFM Section 3.01 





SSS SSS SSS 
LL EEE ene ee ee 


should ensure that the new information technology system produces accurate 
and reliable program-expenditure reports. 


Ministry Response 


The Ministry agrees. The Ministry is committed to ensuring financial 
accountability and will work with Ontario Works service managers to improve 
and enhance the accuracy and reliability of the new information technology 
system. :| 











Participation Agreements 


During the time a person is receiving Ontario Works financial assistance, both that 
individual and all other adult members of that person’s family unit must sign an Ontario 
Works participation agreement. The agreement obligates the signer to take part in one of 
the three types of employment assistance activities described in the Background section of 
this report. A caseworker is to assign each participant the type of activity most appropriate 
for helping that participant find and maintain a paid job. Participation agreements are 
generally valid for a three-month period, after which the participant's progress and needs 
must be reassessed and reflected in a renewed participation agreement, which is valid for a 
further three months. Longer-term participation agreements are permitted for individuals 
with deferred participation requirements, for example, sole-support parents with pre-school- 
age children. 


We reviewed a sample of participation agreements and related summary information about 
them available from the Ministry and noted the following. 


* Many participation agreements were incomplete, often lacking information about an 
individual's educational background and employment history, information that is 
important for assessing the appropriate employment assistance activity. In that regard, 
we noted that: 


- approved participation agreement activities were generally based on client self. 
assessments rather than assessments by caseworkers of recipients’ skills and work 
experience; and 


- many clients were spending extensive time working at an approved activity without 
evidence to indicate that the caseworker had assessed the activity for 
appropriateness. For example, in one instance a participant spent 30 months on 
independent job-search activity and, in another instance, a participant spent more 
than 19 months on a training activity without evidence that the activity had been 
regularly assessed by the caseworker for appropriateness, 


* Many of the most recent participation agreements on file were expired, some for periods 
of between six and twelve months. For example, at one service manager we visited, 
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approximately 70% of all participation agreements had expired, with the result that the 
current circumstances and needs of those participants had not been reviewed and 
reflected in revised participation agreements. 








Recommendation 


To help ensure the Ontario Works program meets its objective of helping 
recipients find and maintain paid jobs, the Ministry should ensure that service 
managers: 


¢ obtain and assess information about each recipient’s educational 
background and employment history to identify the employment-assistance 
activities most appropriate for that recipient; and 


¢ maintain up-to-date participation agreements that accurately reflect 
individuals’ employment-assistance activities and their current 
employment-assistance needs. 


Ministry Response 


The individualized participation agreement tailored to the individual’s needs is 
a cornerstone of the Ontario Works program. With that in mind, the Ministry is 
in agreement with the recommendation and will take measures to assist 
Ontario Works service managers to strengthen business practices. 

















Ministry Monitoring of Service Managers 


The Ministry requires each of its nine regional offices to conduct two types of annual 
reviews of service managers within its jurisdiction to assess compliance with legislative and 
administrative policy requirements: 


e Program-compliance reviews generally consist of a review of a sample of case files to 
assess whether or not they comply with selected program requirements. 


¢ Subsidy-claims examinations are designed to determine whether service-manager claims 
to the Ministry for the Ministry’s share of assistance payments are complete, accurate, 
and based on actual payments to assistance recipients. 


We found that the required compliance reviews for the service managers we visited were 


generally being conducted on an annual basis. We examined a sample of compliance-review 


reports and noted that they identified many of the same concerns we identified during our 
audit as detailed in the earlier sections of this report. However, although the regional offices 
are required to ensure that service managers have taken any necessary corrective action, we 

often found little evidence that such corrective action had been taken. 


We understand that, at the time of our audit, most service managers were not submitting 
monthly claims to the Ministry based on actual program expenditures because of the 
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difficulties with the information technology system noted above. For earlier periods, we 
found that the required annual subsidy-claims examinations had not been conducted for a 
long time or had not been adequately conducted: 


¢ For two of the three service managers we visited, the required annual subsidy-claims 
examination had not been completed for the past four years. 


* For the other service manager, a subsidy-claims examination had been completed 
during the past year, but the scope of the work undertaken was not adequate to 
conclude on the completeness and accuracy of the service manager’s claims because 
insufficient work was undertaken on the basic assistance amounts that represented a 
majority of the total claim. 








ey 


Recommendation 


To help verify that service managers’ subsidy claims are complete, accurate, 
and based on actual payments to recipients, the Ministry should ensure that: 


* actions to correct deficiencies indicated by compliance reviews are carried 
out; 
e subsidy-claims examinations are undertaken annually, as required; and 


e the scope of the work undertaken during subsidy-claims examinations is 
adequate to conclude on the completeness and accuracy of the claim. 


Ministry Response 


The Ministry agrees. The Ministry will undertake subsidy-claims examinations 
annually, and, in addition, deficiencies identified in compliance reviews will be 
acted on and monitored. 








2) 





MEASURING THE EFFECTIVENESS 
OF ONTARIO WORKS 


The Ministry provides its service managers with a comprehensive set of program directives, 
decision-making principles, and program delivery standards. In addition, the Ministry 
enters into binding service agreements with its service managers that include service targets 
that are tied to payments for service. However, the Ministry also needs to systematically 
capture the information necessary to assess program outcomes and hence program 
effectiveness. 


In that regard, we noted that the 2001/02 Business Plan for the Ministry included the 
target of a 5% reduction of the Ontario Works caseload for that year as a measure of the 
program's success. We also noted that the Ministry frequently refers to decreases in the 
number of people receiving benefits as an indication of the program's success. For example, 
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an October 26, 2001 announcement posted on the Ontario government’s Web site 
reported that “7,683 people left the welfare rolls between July and September 2001,” and 


attributed their leaving to the success of the program. 


Significant decreases in the number of recipients are thus considered by the Ministry to be 
an indication that the program is meeting key objectives, including reducing dependency 
and cost to taxpayers. However, the reporting on Ontario Works caseload reductions has 
not differentiated between departures that represent a successful program outcome, such as 
a recipient finding employment as a result of Ontario Works assistance, and other 
departures not related to the program’s operations, such as recipients moving to another 
province or securing other income such as Canada Pension Plan benefits. 


We acknowledge that identifying successful program outcomes is inherently difficult 
because many factors not related to the Ontario Works program can influence the number 


of people leaving Ontario Works. These include: 


¢ improvements in the general economy and the creation of the types of jobs Ontario 


Works recipients are most likely to qualify for; 


¢ local conditions and seasonal factors that influence the availability of jobs in a given area; 
and 


¢ individual commitment and personal initiative to find and maintain a job on the part of 
assistance recipients. 


When recipients leave Ontario Works, caseworkers are to choose a termination code that 
best describes the circumstances. The Ministry’s intent is to use these codes to track why 
recipients have left the program. However, at the time of our audit, these codes could not 
reliably be used for this purpose for the following reasons: 


e The circumstances under which some specific termination codes were to be used were 
not well understood, and codes were used inconsistently by service managers. 


¢ Many of the termination codes used by the caseworkers did not reflect the 
circumstances of the terminations. We found examples at two of the service managers 
we visited. One terminated recipient was incarcerated in a penal institution, but the 
termination code used indicated that termination had occurred because the individual 
had not reported income as required. Another recipient had found a job but was coded 


a “voluntary withdrawal, unknown.” 


° One of the codes—“voluntary withdrawal’—does not provide meaningful information 
about the termination. Over 50,000 terminations between 1998 and 2000 were coded 
as “voluntary withdrawal.” 

















Recommendation 


To determine the effectiveness of the Ontario Works program in helping 
assistance recipients to become self-reliant, the Ministry should: 
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capture and assess the management information necessary to evaluate 
program effectiveness and take corrective action where necessary; and 
e look for ways to make termination codes more useful and ensure that 
service managers understand the circumstances under which specific 
termination codes are to be used and use the codes consistently. 





° | 


Ministry Response 


The Ministry agrees that improvements to the economy and job creation are 
important complements to the effectiveness of the Ontario Works program in 
helping approximately 600,000 people leave the welfare system. By fostering 
personal initiative and providing effective enployment-assistance measures, 
Ontario Works is meeting its objective of helping people to become self-reliant. 


The Ministry agrees that it is important to capture and assess data to evaluate 
program effectiveness and to take corrective action where and when 
necessary. 


The Ministry supports the appropriate and effective use of termination codes 
by Ontario Works service managers. 











OTHER MATTER 
ONTARIO WORKS ADMINISTRATION COSTS 


The Ministry reimburses the 47 service managers for 50% of their direct costs for their 
administration of Ontario Works up to an annual, negotiated maximum amount. We 
understand that each annual, negotiated maximum amount is largely based on historical 
funding rather than on Ontario Works caseloads, even though a 2000 ministry study of 
administration expenses found that caseload size is the most relevant consideration in 
funding decisions. 


Our review of the amounts of administration funding the Ministry provided to all service 
managers as a function of their caseloads showed significant variances, as the following table 
demonstrates. 


Ministry Funding of Administration 
Costs for Service Managers 
per Case, 2000/01 ($) 


Average 





Lowest 





Highest 





Source of data: Ministry of Community, 
Family and Children’s Services 
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As can be seen from the table, the service manager with the highest ministry funding for 
administration costs per case received almost six times as much per case as the lowest-funded 
service manager. We wanted to find out the effect these funding discrepancies had, if any, 
on the number of caseworkers at individual service managers and their resultant caseloads 
and assess the likely impact on program delivery. However, the Ministry could not provide 
us with any information in that regard. 








Recommendation 


To help ensure that Ontario Works program administration is funded 
reasonably and equitably among service managers, the Ministry should 
consider caseload information in its funding decisions. 


Ministry Response 


The Ministry agrees and has already communicated that funding for cost of 
administration will be linked to caseloads. 











Ontario Works Program 61 





| VFM Section 3.01 | 





| VFM Section 3.02 | 





MINISTRY OF FINANCE 


3.02—Corporations Tax 





BACKGROUND 


Generally, the Ontario Corporations Tax Act imposes taxes on all corporations that have a 
permanent establishment in Ontario or that owned and received income from or disposed 
of real property in Ontario as follows: 


¢ Income tax is currently payable at a general rate of 12.5% of taxable income earned in 
Ontario. On the first $280,000 of active business income earned by a Canadian- 
controlled private corporation (CCPC), this rate is reduced by 6.5%. For income 
earned from manufacturing and processing that is not eligible for the CCPC reduction, 
this rate is reduced by 1.5%. 


* Capital tax is payable on a corporation’s taxable paid-up capital (generally 0.3% of 
taxable capital in excess of $5 million). 


* Premium tax is payable on insurance policies written for persons residing in and 
property situated in Ontario at a rate of 2% to 3% of gross premiums. 


* Corporate minimum tax is payable by certain larger corporations or associated groups 
that have annual gross revenues in excess of $10 million or total assets in excess of 


$5 million. 


We note that the Ontario basic corporations income tax rate of 12.5% is, on average, 3% 
lower than the comparable rates in the other provinces. 


Ontario, Quebec, and Alberta are the only three provinces in Canada that require 
corporations to file a provincial corporations tax return directly with the province. The 
Canada Customs and Revenue Agency (CCRA) collects provincial corporations taxes on 
behalf of the other provinces through the federal corporations tax system. The CCRA also 
provides Ontario with daily electronic transmissions of data regarding federal corporations 
tax assessments and reassessments that are reviewed to determine their effect on provincial 
tax owed. 


Certain governmental and non-profit organizations, such as municipalities, Crown 
corporations, and registered charities, are exempt from Ontario corporations tax and are 
therefore not required to file a Ontario corporations tax return. 


For the 2001/02 fiscal year, the province collected approximately $6.6 billion in 
corporations taxes. Although the Ministry was not able to provide us with a summary of the 
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types of taxes making up this total, we noted that, for the 2000/01 fiscal year, the total of 
approximately $9.2 billion in corporations taxes collected by the province consisted of 

$7.2 billion in corporations income tax; $1.3 billion in capital tax; $534 million in 
premium tax; and $139 million in corporate minimum tax. Collectively these corporations 
taxes represent approximately 19% of the total tax revenue collected by the Ministry during 
the year. They also represent a significant increase from the $5.4 billion in corporations taxes 
collected at the time of our last audit of this program in 1996. 


The Corporations Tax Branch (Branch) of the Ministry of Finance has primary 
responsibility for the administration and, together with staff at the four regional tax offices, 
the enforcement of the Ontario Corporations Tax Act. Most of the 770 staff involved in 
corporations tax collection work either in the operations or audit areas of the Ministry. 
Approximately 165 operations staff are located in Oshawa and primarily process tax returns 
and maintain corporations account information. A total of about 500 audit staff are located 
in the Ministry’s head office in Oshawa and four regional tax offices in North York, 
Mississauga, London, and Ottawa. Branch expenditures for the year totalled $45.2 million, 
of which almost 90% was for staff salaries and benefits. 


The Branch’s activities are supported by the comprehensive computer-based Integrated Tax 
Administration System, which facilitates the maintenance of the corporations tax database, 
the processing of tax returns, the issuing of assessments, and the performance of the 
accounting function for taxpayer accounts and provides other information as may be 
required from time to time. 


The Branch is also supported by other ministry branches: the Revenue Operations and 
Client Services Branch receives and processes tax receipts; the Collections and Compliance 
Branch pursues outstanding tax balances and overdue returns; the Special Investigations 
Branch investigates complex cases of suspected non-compliance; and the Tax Appeals 


Branch handles objections and appeals filed by taxpayers. 


AUDIT OBJECTIVE AND SCOPE 


The objective of our audit was to assess whether the Ministry had adequate policies and 
procedures in place to ensure that the appropriate amount of corporations tax is declared 
and remitted by taxpayers in accordance with statutory requirements. 


Our audit work included a review and analysis of relevant ministry files and administrative 
policies and procedures as well as detailed interviews with appropriate staff at the 
Corporations Tax Branch and regional tax offices, the Revenue Operations and Client 
Services Branch, the Collections and Compliance Branch, the Special Investigations 
Branch, and the Tax Appeals Branch. We also obtained relevant information from the 
CCRA and held discussions with staff from other provincial finance ministries. 
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Our audit work covered the period up to March 31, 2002, with emphasis on the policies 
and procedures in place with respect to corporations tax revenues processed in the 2001/02 
fiscal year. Our work was primarily conducted in the period from September 2001 to the 
beginning of the Ontario Public Service Employees’ Union strike in February 2002. We also 
obtained information from telephone calls made both during and after the strike and 
meetings with ministry staff held from the time the strike ended in early May up to June 
2002. 


Our audit was conducted in accordance with the standards of assurance engagements 
encompassing value for money and compliance, established by the Canadian Institute of 
Chartered Accountants, and accordingly included such tests and other procedures as we 
considered necessary in the circumstances. Prior to the commencement of our work, we 
identified the audit criteria that would be used to address our audit objectives. These criteria 
were reviewed and agreed to by senior ministry management. 


During the planning phase of our audit, we reviewed prior audit reports issued by the 
Ministry's Internal Audit Services Branch with respect to the Corporations Tax program to 
determine whether the work of the branch could be relied on to reduce the extent of our 
work. As a result, we relied on the Internal Audit Services Branch’s work on both the 
processing of corporations tax receipts by the Revenue Operations and Client Services 
Branch as well as the collection and write-offs of accounts in default by the Collections and 
Compliance Branch and carried out no further audit work in these areas. We also reviewed 
two other reports by the Internal Audit Services Branch on the Corporations Tax Branch in 
Oshawa and one regional tax office. However, we decided not to rely on these reports since 
the scope of our audit and the approach we took were significantly different from those of 
the Internal Audit Services Branch. 


OVERALL AUDIT CONCLUSIONS 


Many corporations voluntarily declare and remit the correct amount of corporations tax. 
This fact, combined with the strong economy of the late 1990s, contributed significantly to 
the very substantial increase in corporations tax revenues from $5.4 billion in the 1995/96 
fiscal year (reported in our last audit in 1996) to $6.6 billion in the 2001/02 fiscal year 
($9.2 billion in the 2000/01 fiscal year). 


However, where corporations did not voluntarily comply, the Ministry did not have 
adequate policies and procedures in place to ensure that the appropriate amount of 
corporations tax was being declared and remitted by taxpayers in accordance with statutory 
requirements. Although research by the Canadian Tax Foundation on the growth of the 
underground economy indicates that the tax gap—the difference between the amount of 
all taxes actually collected and the amount that should be collected—is increasing, the 
Ministry did not assess or evaluate the extent to which the tax gap affected provincial 
corporations tax revenue. The Ministry also did not assess the risk of significant non- 
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compliance for all corporations that had filed a return and therefore did not select for audit 
those with the highest overall risk. It is our view that the tax gap with respect to provincial 
corporations tax may well be substantial. 


As an example of the extent to which corporations did not voluntarily comply, we found 
that of the 763,000 corporations with active accounts, at least 355,000 corporations—or 
almost half—were in default of filing required returns. The total number of outstanding 
returns for all the years in which those corporations failed to file was at least 930,000. Both 
the number of corporations in default of filing and the number of outstanding returns have 
significantly increased since 1996, when one in five active corporations failed to file and the 
number of outstanding returns was 128,700. These increases are generally attributable to 
the Ministry's reduced efforts in recent years to follow up on overdue returns, the new 
annual EFF filing requirement introduced in January 2000, and the significant increase in 
the number of active accounts. We also noted that: 


e Although information about corporations has been provided by the Ministry of 
Consumer and Business Services (MCBS) to the Branch since the 1970s and has been 
used to automatically update the corporations tax roll on a weekly basis, the Ministry 
does not regularly compare all active registrants in the MCBS database with those in the 
corporations tax roll to ensure that all corporations that are registered with MCBS and 
required to file a corporations tax return continue to be included in the corporations tax 
roll. 


In that regard, we noted that a one-time matching of the MCBS database with the 
corporations tax roll in 1998 identified 40,600 corporations that were active in the 
MCBS database but not included in the corporations tax roll. Although we were 
advised that many of these corporations were subsequently added to the corporations 
tax roll as a result of either a manual review in 1998 or a one-time data synchronization 
project in 1999, the Ministry did not have the information necessary to corroborate that 
assertion. 


We also noted that the Ministry was currently negotiating an agreement with MCBS 
under which the MCBS database and the corporations tax roll would be compared and 


synchronized every six to 12 months. 


¢ The Ministry closed approximately 40,000 accounts in the Integrated Tax 
Administration System (ITAS) that maintains the tax roll, which means that a tax return 
related to any of those accounts would not be expected, and therefore the accounts were 
not included in the Ministry’s statistics on corporations in default of filing; however, 
since these accounts were still registered as active with MCBS and any of them might 
still meet the requirements for filing a tax return, tax returns could have been required. 


With respect to its function of auditing taxpayer returns, we found the following: 


¢ Although the number of discretionary field and desk audits has increased significantly 
since the time of our last audit in 1996 (field audits have more than doubled, and there 
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has been substantial improvement in the number of desk audits), for corporations with 
annual gross revenues of $500,000 and over, the number of desk audits completed was 
only about half of the number planned. For corporations with annual gross revenues of 
under $500,000, very few field or desk audits were performed. This limited the degree 
to which the Ministry could meet its objective of encouraging broad-based voluntary 
compliance. 


Although the Ministry has made a deliberate decision to rely on the CCRA for the 
audit of smaller corporations, it needs to obtain the necessary information to assess 
whether such reliance is justified. 


¢ The Ministry’s audit file documentation did not clearly demonstrate that the necessary 
audit work had been adequately planned and performed and that all work had been 
reviewed and approved by the appropriate manager. When these steps are not taken, 
the Ministry cannot clearly establish whether or not all taxes owed by the taxpayer 
audited have been correctly declared and received. 


¢ The Ministry did not provide sufficient training to auditors to promote their technical 
competence. Insufficient training may limit the ability of auditors to effectively address 
complex technical and industry-specific tax issues. 


We also found that the Branch’s Tax Advisory Unit did not respond to all inquiries from 
other units in the Branch on a timely basis and did not ensure that, where considered 
necessary, all decisions by the Tax Appeals Branch and the courts regarding audit 
assessments and audit assessment reversals were summarized and communicated to the 
appropriate parties. 














eee 


Overall Ministry Response 


We appreciate the Provincial Auditor’s various observations and suggestions 
on the administration of the Corporations Tax program. 


The Provincial Auditor raises valid issues, many of which the Ministry is 
already in the process of addressing. The Ministry will address changes, as 
recommended in the report, in a way that appropriately incorporates risks and 
available resources. 

foo 











DETAILED AUDIT OBSERVATIONS 
TAX RETURN FILING 


Under the provisions of the Ontario Corporations Tax Act, generally all corporations that 
have a permanent establishment in Ontario or that owned and received income from or 
disposed of real property in Ontario must deliver a tax return for each taxation year to the 
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Ministry on or before the last day of the sixth month after the end of the taxation year. 
Alternatively, a corporation may annually file an exempt-from-filing (EFF) declaration if it 
meets all of the following requirements: 


¢ It has filed the required federal corporations tax return. 


¢ It has no Ontario-based taxable income and has no Ontario corporations tax payable for 
the year. 


¢ It was a Canadian-controlled private corporation (CCPC) throughout the duration of 
the year. 


¢ Ithas provided the Ministry with its Canada Customs and Revenue Agency (CCRA) 


business number. 
e = Itis not subject to the corporate minimum tax. 


Tax returns can be filed in any one of three formats. About 80% of filed returns are in 
paper format completed using ministry-approved software; about 10% are in paper format 
completed using ministry-provided forms; and the remaining 10% are filed on disk. 


All returns are received in the Ministry’s Revenue Operations and Client Services Branch. 
This branch ensures that all accompanying payments are deposited and credited to the 
appropriate taxpayer's account on the day of receipt and then forwards the return to the 
Operations area of the Corporations Tax Branch for processing. 


In the Operations area, approximately 100 data fields from each tax return are keyed into 
the Integrated Tax Administration System. If some of the information required to issue an 
assessment is missing or otherwise questionable, the return is referred to the appropriate 
section for follow-up. If the information provided in the return is complete and accurate, an 
assessment is normally issued automatically. 


Reassessments may be issued at a later date as a result of additional information provided by 
the taxpayer or as a result of a ministry audit. The Ontario Corporations Tax Act permits 
reassessments to be issued within four years from the mailing of the original notice of 
assessment for CCPCs and within five years for all other corporations. Thereafter, the 
taxation period becomes statute barred, which means that the return cannot be reassessed. 
However, the four- and five-year deadlines can generally be extended if: a waiver is filed by 
the corporation; the provincial reassessment is to be issued as a result of uncovered fraud; or 
a federal tax assessment or reassessment was issued. 


From January 1, 2000 onwards, the Ministry has been imaging all paper returns processed, 
as well as supporting documentation submitted with disk-filed returns. Imaged returns are 
accessible electronically for use by the appropriate staff of the Corporations Tax Branch and 
regional tax offices. 
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The Corporations Tax Roll 


The Ministry's Integrated Tax Administration System (ITAS) includes a corporations tax roll. 
As of December 2001, the corporations tax roll contained approximately 1.1 million 
registrants, of which 763,000 were active and therefore expected to file an annual return. 
The remaining registrants consisted primarily of corporations whose accounts had been 
closed. 


In general, all corporations that carry on business in Ontario, regardless of where they are 
incorporated, must register with the Ministry of Consumer and Business Services (MCBS). 
In addition, corporations must advise MCBS of significant changes in their circumstances 
(for example, if a corporation amalgamates with another corporation, dissolves, or ceases to 
operate in Ontario). Since the 1970s, information about corporations provided to MCBS 
has been used to update the Ministry’s corporations tax roll. Currently, on a weekly basis, all 
changes in a corporation’s information that MCBS has received are electronically 
transmitted to the Ministry, and I'TAS automatically updates the tax roll. For example, in the 
case of an amalgamation of corporations, [TAS opens a new account for the newly 
amalgamated corporation that is linked to the predecessor corporation accounts, which are 
then closed. The corporations tax roll may also be updated manually, in cases such as the 
following: 


* when a tax return is received from a corporation that is not already on the tax roll; and 


* when electronic transmissions of data regarding federal corporations tax assessments and 
reassessments are received from the CCRA for corporations that are not already on the 
tax roll. 


However, we noted that the Ministry does not regularly compare all active registrants in the 
MCBS and ITAS databases and ensure that all corporations registered with MCBS that are 
required to file a corporations tax return continue to be included in the corporations tax roll 
in ITAS. A one-time matching of the MCBS database with the ITAS tax roll in 1998 found 
that there were 40,600 corporations active in the MCBS database that were not set up in 
I'TAS. We understand that, as a result of a one-time data synchronization project in 1999, 
MCBS data were used to automatically update the corporations tax roll in ITAS. However, 
information describing how 39,900 of the discrepancies identified in 1998 were resolved 
and information about the results of the data synchronization was not available at the time 
of our audit. 


We understand that the Ministry is currently negotiating an agreement with MCBS under 
which the MCBS database and the ITAS tax roll would be compared and synchronized 


every six to 12 months. 


We also noted that the Ministry had manually closed about 40,000 accounts in ITAS that 
were still registered as active with MCBS and might still meet the requirements for filing a 
tax return. However, the result of closing the accounts was that these corporations would no 
longer be expected to file a tax return and would not be included in the Ministry’s statistics 
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on corporations in default of filing tax returns. We understand that in the case of about 
15,000 of these accounts, the corporations owed money to the Ministry, and the accounts 
were closed in part to permit the Ministry to write off the outstanding amounts. However, 
none of the accounts should have been closed, since, as noted, all were still registered as 
active with MCBS, and insofar as any of the accounts might still meet the requirements for 
filing a tax return, tax returns could have been required. 


Corporations in Default of Filing 


In any year, a certain number of active corporations are in default of filing tax returns, and 
many of these corporations have failed to file returns over a number of years. For example, 
ministry records indicated that in 2001, 355,000 corporations, from a total of 763,000 
corporations with active accounts, were in default of filing returns. The total outstanding 
returns for all the years in which those corporations failed to file was 930,000. The 
following table shows the number of active corporations in default of filing returns each year 
as well as the overall number of returns not filed over the years the corporations have been 


in default. 


Corporations in Default of Filing Returns, 1996-2001 













1996 464,138 92,990 128,700 
1997, 545,530 147,700 200,000 


1998 563,317 185,000 268,000 















1999 605,000 220,000 365,000 
2000 685,000 265,000 690,000 
2001 763,000 355,000 930,000 


Source of data: Ministry of Finance 








When the default and returns outstanding data are expressed as a percentage of total active 
accounts, a dramatic increase since the time of our last audit in 1996 is seen, as in the 


following line graph. 





69 


Corporations Tax 


VEM Section 3.02 








Auk. Section 3.02 | 





Corporations in Default of Filing Returns 
as a Percentage of Total Active Accounts, 1996-2001 
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Source of data: Ministry of Finance 


The Ministry attributed the increase in the number of corporations in default of filing in 
2000 and 2001 in part to the new annual EFF filing requirements, which came into effect 
on January 1, 2000, and to the absolute increase in the number of active corporations. 
However, since many of the corporations that were presumed to be EFF eligible did not 
meet all of the requirements for EFF status, they really are in default. As a result, the 
Ministry's default statistics for 2000 and 2001 correctly include these corporations, and the 
default statistics for 1999 and earlier years are most likely significantly understated. 


Overall, the increases in the number of corporations in default of filing since the time of our 
last audit are largely attributable to the Ministry’s reduced efforts in recent years in following 
up on overdue returns. Examples of those reduced efforts include the following: 


* Default notification letters requesting that outstanding returns be filed have not been 
regularly sent since 1995. 


The threat to cancel a corporation’s charter and seize its assets as a deterrent to chronic 
non-filers has not been used since 1997. 


We also understand that the Ministry’s efforts in this area are being hampered by 
inaccuracies in ITAS; for example, we were advised that the addresses of 71,000 


corporations in default of filing returns are currently being traced so that notices requesting 
the required returns can be issued. 
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Another means of encouraging a corporation to file an overdue return is to issue it an 
arbitrary assessment of taxes owing based on the most recently filed return. However, we 
found that the Ministry seldom uses this measure. 


We noted that in February 2001, the Ministry established a unit, presently staffed at 15 
employees, but ultimately to be staffed at 30 employees, to follow up on corporations that 
failed to file a required return. The initial focus of the unit was on corporations that had 
filed a recent return but had at least one return outstanding for an earlier year. By the end 
of December 2001, approximately 2,700, or 80%, of the 3,400 corporations that the 
Ministry followed up on filed the required returns. These returns generated $2.6 million in 
self-assessed tax. In addition, 30 corporations were arbitrarily assessed taxes totalling 
$130,000. The Ministry advised us that it hopes to collect $115 million in additional taxes 
from all corporations in default of filing a required return by the end of the 2003/04 fiscal 
year. 

We estimated that, using the Ministry’s current processes, once the unit is fully staffed it will 


take a significant number of years to follow up on and clear all corporations currently in 


default of filing a required return. 


Recommendation 


To help ensure that all required corporations tax returns are received and 
processed and that the appropriate amount of taxes is collected, the Ministry 
should: 








e regularly compare the corporations registered in the Ministry of Consumer 
and Business Services database with those in its own corporations tax 
database and investigate and resolve discrepancies on a timely basis; 

e make better use of available tools to enforce compliance by defaulting 
corporations; and 

e assess whether additional resources and procedures are warranted to 
follow up on all outstanding returns and ensure that those returns are 
appropriately submitted. 


Ministry Response 


The Ministry has to date implemented or taken the appropriate steps to initiate 
the following measures in these areas: 





¢ Weekly reports from the Ministry of Consumer and Business Services 
database will be analyzed in comparison to the corporations tax database. 

e Another comprehensive synchronization of the two databases will take 
place in the fall of 2002. 

¢ Over the past year, the Ministry has applied additional staff and information 
technology (IT) resources to more effectively follow up on corporations 
with outstanding returns; the corporations tax database is being matched 
with federal data to further enhance current risk assessment techniques. 
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¢ Selective default notices have and continue to be issued for higher-risk 
corporations tax returns that are overdue. 


¢ Charter cancellation will be enforced for non-compliant corporations. 


We will continue to assess whether additional staff and IT resources are 
required to adequately address the volume of accounts with outstanding tax 
returns or annual declarations due and to validate their exempt-from-filing 
status. This would also include reviewing the practice and criteria for selective 
| sending of default notices. 











ai] 


TAX RETURN PROCESSING 


The Ministry has an informal guideline in place that requires a submitted tax return to be 
processed and an assessment to be issued within three to five days of receipt of the return. 


We noted that the vast majority of tax returns that are submitted to the Ministry include all 
of the required information. We found in our review of a sample of such returns that the 
returns were processed and the assessments were generally issued within three to five days of 
receipt of the return, as required by the Ministry’s guideline. 


However, filed tax returns cannot be completely processed in the following circumstances: 


* The filed return either lacks required information, or some information provided 
appears to be questionable and therefore requires verification before the return can be 
completely processed. 


* The taxpayer filing the return is in default of filing a previous year’s return. 


Returns that cannot be processed are referred to the appropriate unit for follow-up. 
However, in many cases there was no evidence that the Ministry did follow up on the 
missing information, and therefore these returns were often not processed on a timely basis. 
For example: 


* As of December 2001, the Ministry held but could not process 8,000 filed returns 
because the taxpayers were in default of filing a previous year’s return. Approximately 
10% of these returns had been held for over a year. We were advised that the Ministry 
had not made any attempts since September 2001 to obtain the missing previous years’ 
tax returns for approximately 6,500 of these returns. 


* As of December 2001, the Ministry held approximately 19,000 returns that could not 
be processed because they lacked some of the required information or because 
information provided had to be verified. Many of these had been held for an extended 
period of time—in some cases for over a year and for as long as four years. We found in 
our review of a sample of these returns that in many cases it was not clear what if 
anything had been done by staff at the units to which the returns had been assigned to 
obtain the required information. 
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In August 2001, the Ministry assigned one person to follow up with the units to which 
the returns had been assigned. 










Recommendation 





To ensure that filed returns can be processed and that the correct amount of 
tax is collected or refunded on a timely basis, the Ministry should follow up on 
missing information or, when necessary, verify information provided in returns 
on a timely basis. 


Ministry Response 





The Ministry has been following up on a monthly basis on the outstanding 
work items. Steps will be taken to ensure more timely follow-up on information 
missing from taxpayers’ returns, with enhanced tracking and recording of staff 
efforts in this regard. In addition, we will continue to review and implement 
additional procedures as required to more effectively manage this process. 





MINISTRY AUDIT ACTIVITIES REGARDING 
TAXPAYERS 


The objectives of the Ministry’s audit activities regarding taxpayers are: 


¢ to ensure that the corporations selected for audit have declared and remitted the correct 
amount of tax; and 


e through the broader taxpayer population’s awareness of the Ministry’s audit program, to 
encourage the broadest possible degree of voluntary compliance. 


To achieve these objectives, the Ministry conducts three types of audits: 


¢ discretionary field audits—This type of audit is generally limited to corporations 
reporting over $7 million in annual gross revenues. Discretionary field audits are 
conducted at the taxpayer’s premises and may include a detailed examination of a 
corporation’s underlying records. For 2000/01, discretionary field audits generated an 
additional $225 million in tax assessments. 


¢ discretionary desk audits—This type of audit is generally limited to corporations 
reporting between $500,000 and $7 million in annual gross revenues. Discretionary 
desk audits are primarily conducted in two of the Ministry’s regional tax offices and 
generally include a detailed review of the tax return. In many instances, additional 
information is requested from the taxpayer by telephone or through correspondence. 
For 2000/01, discretionary desk audits generated an additional $9 million in tax 
assessments. 





Corporations Tax 73 


| VEM Section 3.02 | 








| VFM Section 3.02 | 


NN 


* nominal desk audits—This type of audit is a mandatory review of a corporation’s returns 
that includes: reviews of requests for loss carry-backs (that is, requests to apply losses 
incurred to retroactively decrease the taxable income the corporation reported in any of 
the three prior years to allow the corporation to claim a refund of some prior years’ tax 
payments); reviews of requests for refundable Ontario tax credits and amended returns; 
and the review of federal corporations tax assessments and reassessments for Ontario tax 
implications. For 2000/01, nominal audits resulted in $85 million in additional tax 
assessments and tax reductions totalling $131 million, most of which were a result of loss 
carry-backs. 


Ontario corporations may also be subject to audit by the CCRA of the federal government. 
However, except for the sharing of federal corporations tax assessments and reassessments as 
noted in the bullet above, there have been only limited attempts to co-ordinate audit 
coverage between the two jurisdictions. 


Audit Coverage 


‘To meet its two objectives of ensuring that corporations selected for audit have declared and 
remitted the correct amount of tax and encouraging broad-based voluntary compliance, the 
Ministry has established audit coverage goals by type of audit, as shown in the following 
table. 


Planned versus Actual Audit Coverage by Type of Audit, 2000/01 

















9, 
vel bane 
Type of Audit Corporations Audie” 
Eligible for Audit Completed 
Discretionary field audit 17,600 94 
Discretionary desk audit for corporations reporting 
$500,000 and over ee a 
Discretionary field audit and discretionary desk audit 
for corporations reporting less than $500,000 sonar Ue n/a 
potentially all 

Nominal desk audit 763,000 active 215 


corporations 


* No audits are planned for these corporations, and a minimal number were conducted based 
primarily on referrals from the audits. 


Source of data: Ministry of Finance 


As the table indicates, the Ministry completed 94% of the planned number of discretionary 
field audits of corporations with annual gross revenues of over $7 million. The largest of 
these corporations—those with annual gross revenues exceeding $100 million—have 
generally been audited on a three- to four-year cycle as planned. We also noted that the 


eee 
74 2002 Annual Report of the Provincial Auditor of Ontario 





number of discretionary field audits completed has more than doubled since the time of our 
last audit in 1996, due in part to the doubling of the number of field auditors and 
increased audit coverage of smaller corporations with annual gross revenues of between 

$7 million and $25 million. The low coverage of these latter corporations was a concern in 
our 1996 report. 


Although the Ministry conducted only about one half of the planned number of 
discretionary desk audits of corporations reporting annual gross revenues of $500,000 or 
more, the number of audits completed still represents a substantial improvement since the 
time of our last audit in 1996, when only about 200 such audits were completed. No 
specific number of audits was planned for that year, and the low coverage at that time was 
primarily due to the fact that most desk auditors were redeployed to the return-processing 
function at that time. 


We also noted that corporations with annual gross revenues of less than $500,000, which 
represent about 87% of the total number of corporations on the tax roll, are not normally 
audited unless specific issues are identified with respect to a particular corporation. This is of 
particular concern because many of these corporations are owner-managed and therefore 
may have limited internal controls and financial expertise, which could make them 
susceptible to compliance deficiencies. In that regard, we noted that, in a study conducted 
by a professor at a Canadian university of both U.S. and Canadian empirical evidence 
regarding components of the tax gap, it was found that the percentage of unreported taxes 
as a function of total tax liability was highest for small corporations. Roughly one half of the 
amount in unreported taxes was attributable to unreported income and one half to 
overstated expenses and deductions. 


We also noted that a significant proportion of CCRA audits are of smaller corporations, 
many with revenues of less than $300,000. We were advised that the Ministry in effect 
relies on the CCRA to audit these small corporations and has made a deliberate decision 
that the audit coverage of these corporations should be left to the CCRA. While this would 
appear to be a reasonable approach to the audit of small corporations, the Ministry did not 
obtain specific information about the CCRA small-business audit program on which to base 
such reliance. For example, it did not obtain and monitor information about the CCRA 
small-business audit such as audit coverage, audit selection criteria, the nature and extent of 
work performed, and the results obtained. 











Recommendation 


In order to meet its objectives of ensuring that corporations selected for audit 
declare and remit the correct amount of tax as well as encouraging broad- 
based voluntary compliance with the Ontario Corporations Tax Act, the 
Ministry should: | 


e conduct the planned number of discretionary desk audits of corporations 
with annual gross revenues of between $500,000 and $7 million; and 
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* consider the advisability of auditing, based on assessed risks,more 
corporations with annual gross revenues of under $500,000. 


Alternatively, if the Ministry continues to rely on Canada Customs and Revenue 
Agency (CCRA) audits, the Ministry should obtain and monitor specific 
information about the CCRA small-business audit program so that it can 
assess whether the program is meeting its expectations and whether the 
Ministry’s reliance on the CCRA audits is justified. 


Ministry Response 


The Ministry will set realistic targets for the number of discretionary desk 
audits to be undertaken. We will continue to collect, monitor, and analyze data 
on the actual number of completed audits against the budgeted figures on a 
monthly basis and take appropriate action to ensure the planned number of 
audits are completed. 


The Ministry's reliance on CCRA audits of the smallest corporations on the tax 
roll is intended to achieve efficient use of ministry resources and minimize the 
compliance burden for these smaller corporations. 


However, the Corporations Tax Branch will obtain additional information from 

the CCRA about its small business audit program to better assess the ongoing 
effectiveness of this approach and make such changes as are indicated by that 
review. a 











Discretionary Field and Desk Audits 


AUDIT SELECTION 


The process whereby individual corporations are selected for audit is a critical aspect of 
ensuring that the Ministry’s audit function meets its dual objectives efficiently and 
effectively. Both to ensure that the Ministry obtains the correct amount of tax owing and to 
encourage broad-based voluntary compliance, it would be expected that the selection 
process would target those corporations with the highest assessed risks of significant non- 
compliance and that the selection process would target a sufficiently diverse range of 
corporations. 


We understand that prior to the implementation of ITAS in 1995, the Ministry used to 
have a system in place that automatically ranked all processed returns using a risk-based 
point system and made selections from the ranked returns. Such a system is no longer in use. 
Instead, senior audit managers in each of the four regional offices and in the Corporations 
Tax Branch randomly select groups of 20 to 50 corporations from a listing of the portion of 
the tax roll assigned to them. They then conduct a cursory review of the tax returns of each 
of these corporations and select on the basis of their own judgment 10 to 20 of them fora 


discretionary field audit. 
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A similar process is followed for selecting corporations for discretionary desk audit, although 
the initial random selection is made from a listing of corporations that meet certain criteria 
(for example, corporations that have claimed eligible investments for capital tax purposes or 
corporations that are associated with other corporations). 


Our concerns with respect to the current selection process are as follows: 


¢ Because the initial selection of 20 to 50 corporations is random, there is no assurance 


that: 


- those corporations with the highest risks of non-compliance are included in the 
initial selection and therefore will be considered for selection for audit; and 


- asufficiently diverse range of corporations, particularly in terms of type of industry 
and geographic location, is selected to satisfy the ministry objective of encouraging 
broad-based voluntary compliance (for example, we were advised that some 
managers limit the number of corporations they select for field audit that have head 
offices located a long distance from the regional office and thereby minimize travel 


time and costs). 


e The subsequent review of a corporation’s tax return is not documented. Also, specific 
reasons for ultimately selecting a particular corporation for audit are not given, and 
therefore the basis for selection cannot be determined. Rather, only a general reason for 
the corporation’s referral is provided, such as “provincial allocations,” “capital tax,” or 


(<4 © >>) 
general review. 


¢ Some of the senior audit managers we talked to readily acknowledged that, despite their 
best efforts, many of the selections of corporations they made would not result in a 
significant reassessment, while, as already mentioned, other corporations with high risks 
of non-compliance are not necessarily identified. We found that almost one-half of all 
audits completed during 2000/01 resulted in nil or minimal reassessments of taxes 


owed. 


We were advised that the Ministry was in the process of developing general guidelines for 
audit managers to consider during the initial file reviews as well as a checklist that would 
document their file review. 


In contrast to the Ministry’s informal audit selection process, we noted that, for medium- to 
large-sized corporations, the CCRA uses a detailed risk-assessment process to assess potential 
workload. This process includes the use of risk-assessment checklists specific to particular 
industries and consultations with industry specialists who advise on issues and emerging 
trends as they arise. This detailed risk assessment is also used to identify specific audit issues 
and continues throughout the progress of the audit to ensure that only material issues are 


pursued. 
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For smaller-sized corporations, the CCRA uses computerized information-matching 
procedures and applies qualitative and quantitative statistical analyses to identify and select 
for audit those corporations with a high probability of non-compliance. 


We were advised that the Ministry’s Tax Revenue Division initiated an Audit Selection 
Project in July of 2001 that deals with all taxpayer audit activities for all the different types of 
taxes collected by the Ministry. The objective of the project is to identify and document the 
current audit selection methods used by branches of the Ministry involved with tax 
collection and the Ministry’s regional tax offices in order to identify best practices and assess 
the feasibility of centralizing the audit selection function. 





Recommendation 


To ensure that the Ministry’s audit function meets the Ministry’s compliance 
and tax collection objectives efficiently and effectively, the Ministry should 
ensure that: 


e its audit selection process assesses the risk of significant non-compliance 
for all corporations and selects those with the highest assessed risk of 
significant non-compliance; and 


¢ it monitors the range of corporations selected for audit to ensure that it is 
sufficiently diverse in terms of industry type and location to encourage 
broad-based, voluntary compliance. 


Ministry Response 


While the Ministry's current selection process incorporates consideration of 
risk of non-compliance, we will further investigate the use of a more formalized 
risk-based approach to further enhance the selection process for both field 
and desk audits. 


The Ministry is also in the process of establishing a new Audit Control and 
Analysis Unit in the Corporations Tax Branch that will maintain and analyze a 
database of all assigned and completed audits. The unit will ensure that the 
range of corporations selected for audit is sufficiently diverse in terms of 

| industry and location. 











AUDIT WORK COMPLETED 


For auditors conducting discretionary field and desk audits, the Ministry provides an audit 
manual containing guidelines for audit performance standards that “outline best practices 
and procedures to assist and direct auditors in fulfilling the requirements of the job.” As the 
manual states, “the guidelines are described in a less structured way so as to provide for 
individual creativity, flexibility, and use of judgment within the requirements of the 
position.” For example, with respect to audit program preparation, the manual states that 
the nature and extent of the program to be developed depends on the particular 
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circumstances of the corporation being audited and then identifies the main factors that 
influence audit program development. 


As a result, in practice, individual auditors have considerable discretion with respect to how 


the requirements of the guidelines are fulfilled. 


We found in our review of a sample of both field and desk audit files completed during the 
past two years that all proposed audit adjustments arising from areas audited and resultant 
reassessments issued were correctly determined and supported by the work performed. 
However, we also noted a number of concerns that ultimately brought into question 
whether or not all areas where audit work could be justified had been audited and 
consequently whether or not all taxes properly owing had been assessed and paid. The 
concerns noted included the following: 


¢ In most instances, we found no evidence that the auditor had assessed all of the areas of 
potential risk and identified and prioritized the audit work necessary to mitigate the 
risks. In addition, some significant areas of risk such as transfer pricing or related party 
transactions were not identified and pursued, since it is the Ministry’s view that it is not 
in a position to do so for several important reasons. For example, the Ministry assumes 
that it does not have the necessary authority to assess transfer pricing issues because it 
does not have access to the competent authority process that is only available at the 
federal and international levels. The Ministry also noted that it cannot possibly dedicate 
the resources needed to adequately address complex international tax issues. We were 
advised that, instead, the Ministry relies on the CCRA with respect to the audit of 
transfer pricing and related party transactions for Ontario-based corporations. However, 
the Ministry lacked the information about CCRA audits necessary to provide assurance 
that such reliance was justified. 


¢ Audit file documentation was generally in need of improvement. Most files reviewed 
did not contain audit programs detailing the nature and extent of the work planned 
and performed. In approximately one-third of the files we reviewed, we identified at 
least one issue that should have been pursued but was not, and the reason for not doing 
so was not documented. 


¢ In most cases, we found little evidence of managerial input during the audit planning 
stage. Similarly, with respect to the work performed, except for sign-offs on the front of 
the file, there was often no other evidence of detailed managerial review and approval, 
such as review notes. 


We also noted that, to ensure consistent application of its policies and procedures, the 
Ministry formed a Quality Review Unit in 1999/2000 to review completed desk and field 
audit files at all regional tax offices and at the Branch. In the 2000/01 fiscal year, the unit 
conducted reviews of field audit files at two regional tax offices. The results of those reviews 
identified similar concerns to ours with respect to audit planning, file documentation, and 
managerial review. 
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Recommendation 


To ensure that all necessary audit work is completed satisfactorily and that the 
work performed clearly establishes whether or not taxes owed have been 
correctly declared, the Ministry should ensure that: 


e auditors identify and assess all potential risks of non-compliance by the 
corporation selected for audit and identify and prioritize all the audit work 
that needs to be performed; 


e where reliance is to be placed on the work performed by the Canada 
Customs and Revenue Agency (CCRA), it obtains the necessary 
information about the CCRA audit activities to provide assurance that such 
reliance is justified; 

e auditors use detailed audit programs that clearly indicate the nature and 
extent of audit work proposed and actually performed; and 


° managers adequately document their input at the planning stage of an audit 
as well as their review and approval of the work performed. 


Ministry Response 


The Ministry will take appropriate measures to ensure that its current 
standards are being met consistently in all of the Ministry’s office locations 
where corporations tax audits are undertaken. The Ministry will also review its 
current field and desk audit standards and make further enhancements to 
these standards where appropriate. 


The Ministry will also examine those areas where reliance is placed on the 
work of the CCRA, such as international transfer pricing issues, and will 
consider if there is an ongoing need to obtain more information about the 
CCRA’s audit programs in those areas. 











Nominal Desk Audits 


Nominal desk audits generally consist of a review of returns that have been amended or 
contain requests for loss carry-backs or the review of a corporation’s federal assessment or 
reassessment data to determine their effects on provincial tax owed. 


Since April 1, 2001, the Ministry of Finance has been receiving daily electronic 
transmissions of data regarding federal corporations tax assessments and reassessments issued 
by the CCRA. A provincial corporations tax reassessment, if warranted, must be issued 
within 365 days of receipt of the federal assessment or reassessment data ifa provincial 
return has been filed for that year and if the return is otherwise already statute barred as 
defined above in our section on “Tax Return Filing.” 
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However, we understand that there are four circumstances under which a review of federal 
assessment or reassessment data is not automatically assigned and therefore cannot proceed, 
as follows: 


¢ The corporation in question does not have a CCRA account number in ITAS. 
¢ The corporation’s status in ITAS is inactive. 
¢ The corporation’s federal and provincial taxation periods as filed do not coincide. 


¢ The corporation has filed a provincial EFF declaration or is in default of filing the 


required provincial tax return. 


We found that, when any of these circumstances arose, the Ministry was often not timely in 
investigating the situation and determining whether the appropriate amount of provincial 
corporations tax had been declared. The result of not reviewing the federal assessment and 
reassessment data in these circumstances for their effects on provincial tax owed can be that 
provincial tax is not collected on a timely basis and, as in the case of the 22,000 federal 
assessments or reassessments discussed below that have not yet been reviewed, that the 
assessments and reassessments become statute barred and provincial tax revenue is lost. For 


example, for the period of April 2001 to February 2002, we found the following: 


¢ The Ministry received over 5,300 federal corporations tax assessments or reassessments, 
representing $622 million in federal taxable income, for corporations that had filed a 
provincial EFF declaration or were in default of filing the required provincial tax return. 
We were subsequently advised that, between the time that we brought these items to the 
Ministry’s attention in February 2002 and June 2002, the Ministry resolved 
approximately 1,300, or about one-quarter, of these items but assessed no additional 
provincial tax. 


¢ The Ministry received 41,000 federal assessments or reassessments for corporations that 
did not have a CCRA account number in ITAS. We noted that, between October 
2001 and the end of our audit in June 2002, the Ministry reviewed approximately 
17,000 of these assessments or reassessments and identified approximately 600 for 
potential audit. Of these, 70—with potential recoveries of $870,000—have been 
assigned for audit, but the audits have not yet been completed. Unless the remaining 
24,000 federal assessments or reassessments are soon reviewed and, where warranted, 
assigned for audit, they could become statute barred. 


Recommendation 


To ensure that provincial corporations tax assessments and reassessments 
resulting from federal assessments or reassessments are issued on a timely 
basis and do not become statute barred, the Ministry should ensure that it 
reviews all federal corporations tax assessments and reassessments and 





completes any required audit work to determine provincial corporations tax 
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Ministry Response 






Steps are being undertaken to accelerate the review of federal corporations tax 
assessments and reassessments. Staff recruitment has been initiated inthe _ 
Desk Audit Section of the Corporations Tax Branch to bring staff back up to 
the approved complement level. In addition, desk audit staff in the two regional 
tax offices are assisting with the processing of these adjustment files. 












TRAINING NEEDS 


In order to keep abreast of the numerous legislative tax changes that occur and be able to 
adequately audit and assess the reliability of information provided by corporations’ 
increasingly complex accounting systems during tax audits, ministry audit staff need to be 
properly trained. ‘Training of staff needs to be ministry-wide so that audits are conducted 
professionally and consistently across the province. 


Over the past five years, a great number of auditors were hired for both discretionary desk 
and field audits. Since the desk audit positions in particular are considered to be entry-level, 
some of the individuals hired for these positions did not have extensive audit or taxation 
backgrounds. The Ministry’s initial training of auditors consisted of an introductory course 
that covered the basic requirements of the audit function, the workings of ITAS, and 
general legislative requirements. 


As for ongoing ministry-wide training, we were advised that, in the two-year period of 2000 
and 2001, only a total of three days of ministry-wide training was provided. The training 
covered areas such as legislative changes and the impact of results of tax appeals and 
taxpayers’ objections. Training was not provided in advanced audit and technical legislative 
requirements. 


We found in our visits to several regional tax offices that each office also provides its own on- 
the-job or informal internal training on issues specific to the office. However, information 
available was not sufficient to determine the nature and extent of such training provided or 
to assess its adequacy and effectiveness. 


In addition, the field auditors we interviewed informed us that the training received overall 
was insufficient to provide them with the detailed technical knowledge they would need to 
conduct comprehensive tax audits specific to particular types of industries. Inadequate 
training may be one of the reasons why some audit issues were not examined in the files that 
we reviewed. 


In addition to being advised of auditors’ concerns regarding the extent of ministry training, 
we were also advised by many auditors that the Ministry often has not funded training 
opportunities pursued by individual auditors to increase their tax knowledge base. Such 
opportunities include membership in the Canadian Tax Foundation and participation in 
the Canadian Institute of Chartered Accountants’ in-depth tax course. 
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Recommendation 


To help enable field and desk auditors to effectively and consistently address 
corporations tax issues and thereby improve tax collection efforts, the Ministry 
should: 


¢ ensure that sufficient training that adequately addresses both technically 
complex issues and industry-specific high-risk areas is provided for both 
field and desk auditors; and 


¢ consider funding, on an individual basis, training initiatives that would 
increase the individual auditor’s knowledge base. 





Ministry Response 


The Ministry recognizes the importance of staff training and has recently 
adopted enhanced training as a strategic priority. To this end, job-specific 
technical training plans were developed for all audit positions in the division. 


A training unit was established earlier this year in the Corporations Tax Branch 
to enhance current training materials and.develop additional technical training 
materials for both audit and other staff dealing with corporations tax issues in 
the Branch and in the regional tax offices. 





The Ministry will review its current provision of funding for training on an 
individual basis for auditors and other staff with a view to ensuring training 
opportunities where they are most required on a priority basis. 





TAX ADVISORY SUPPORT 
Requests to the Tax Advisory Unit 


One of the key responsibilities of the Tax Advisory Unit is to recommend necessary changes 
to the existing legislation and prepare and communicate, for all units within the Branch, 
legislative interpretations and rulings for taxpayers. [he majority of the interpretation 
requests come from the field audit area. The requests must be formally directed to the Tax 
Advisory Unit on a Tax Advisory Enquiry Report form. Upon receipt of these requests, the 
Tax Advisory Unit records the date the request was received, the nature of the request, the 
staff assigned to complete the request, and the status of the request. We understand that, 
during the 2001/02 fiscal year, the Tax Advisory Unit provided 54 advance rulings and 64 


written interpretations. 


Although, according to the Tax Advisory Unit’s senior management, there is no requirement 
as to how quickly these requests must be completed and returned to the party requesting the 
information, management believes that it is reasonable for those making a request to expect 
a resolution within 90 days of the Advisory Unit receiving the request. However, 
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management staff indicated that, for the past year and a half; most requests have taken from 
six months to a year to complete. 


Our review of the outstanding request log as at March 2002 showed that 25 internal 
requests remained outstanding, 18 of which were older than 90 days and some of which 
had been outstanding since November 1998. Management staff at the Tax Advisory Unit 
informed us that they were aware of the long outstanding requests but were unable to 
address them due in part to staff vacancies in the area. Some positions had been vacant since 
late 2000. At the time of our audit, the Tax Advisory Unit was in the process of hiring for 
these positions. 


Communicating the Decisions of the Tax Appeals 
Branch 


Significant decisions made by the Tax Appeals Branch that, for example, result in audit 
assessment reversals, as well as court decisions regarding audit assessments, are sent to the Tax 
Advisory Unit. In order for the decisions to assist staff in future audit work and assessments, 
where warranted, they are to be summarized and communicated to all regional tax offices 
through channels such as interpretation or information bulletins or through changes to 
existing policies. 


Management staff at the Tax Advisory Unit informed us that they have not formally 
reviewed and, where warranted, summarized 329 tax appeals decisions made between July 
2000 and March 2002. The delay was again due to staff vacancies. At the time of our audit, 
management staff were in the process of hiring one person to perform the function of 
reviewing and summarizing the decisions. In the meantime, auditors would not be informed 
of tax appeals decisions that could impact on their audit approach and their ability to ensure 
that assessment and appeals decisions are consistently considered in the work the auditors 


undertake. 





| Recommendation 


To provide good taxpayer service and effectively utilize audit resources, the 
Tax Advisory Unit of the Corporations Tax Branch of the Ministry should: 


e establish a standard completion time for formal requests for tax advisory 
services; 

¢ address all formal legislative and interpretational requests from regional 
tax offices within the standard completion time established; and 

¢ summarize and, where warranted, communicate all tax appeals decisions to 
all relevant parties in the appropriate manner and ona timely basis. 
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Ministry Response 


In November 2001, the Tax Advisory Unit of the Corporations Tax Branch was 
restructured to improve and enhance the range of services provided. Staffing 





levels are being increased. The Ministry is also currently consulting with its 
regional tax offices to develop a reasonable response time for standard tax 
advisory requests. 


The Tax Advisory Unit will expedite the summary and review of significant tax 
appeals decisions. Corporations tax staff were advised and will be advised on 
a more timely basis of decisions and tax cases that significantly impact on the 
Ministry’s assessing policies and practices in order to assure timely 
processing. 








THE TAX GAP 


As noted previously, one of the Ministry’s key objectives is to encourage the highest possible 
degree of voluntary compliance from taxpayers and thereby reduce the tax gap (the 
difference between the amount of all taxes actually collected and the amount that should be 
collected). To know whether this objective is being achieved, the extent of the tax gap must 
be determined and monitored over a period of time. 


The greatest contributor to the tax gap is what is commonly called the “underground 
economy.” The underground economy can be understood to consist of those economic 
activities that are hidden from public authorities to avoid taxation. While we appreciate that 
attempts to estimate the size of the underground economy are fraught with difficulties due 
to differing definitions and assumptions, the latest evidence points to an increasing amount 
of underground economic activity that is resulting in an increasing tax gap. A research study 
published in 2002 by the Canadian Tax Foundation found that the underground economy 
in Canada grew steadily between 1976 and 1995 from about 3.5% to 15.7% of the gross 
domestic product (GDP). The study estimates that those percentages translate to $2 billion 
in missing tax revenue in 1976 and all of $44 billion in missing revenue in 1995. The 
underground economy has not only increased in absolute value, but it has grown 
dramatically relative to the regular economy. The study estimated the value of the 
underground economy in 2002 at close to 20% of GDP. In 2001, Ontario’s GDP 
represented about 40% of the total Canadian GDP. 


In the absence of any ministry assessment and evaluation of the extent to which the tax gap 
affects provincial corporations tax revenue, it is our view that the tax gap with respect to 
provincial corporations tax may well be substantial. 


Concerns with the growth of the underground economy have prompted the CCRA to 
launch a number of initiatives. For example, a recent initiative introduced contract 
reporting, which requires general contractors to report to the CCRA both whom they hired 
and the amount paid them. In addition, an initiative called the Community Awareness 
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Program sends a team of auditors into the community to increase taxpayer awareness and to 
respond to any taxpayer questions or concerns. 


The Ministry's Macroeconomic Analysis and Policy Branch is responsible for developing 
revenue forecasts and estimating potential tax loss attributable to tax evasion. Our discussion 
with staff from that branch indicated that no studies on the underground economy as it 
relates to corporations income taxes had been conducted in recent years. However, we 
believe it is critical for the Ministry to identify any areas of the economy subject to high risk 
of tax evasion and thus contributing to the tax gap in order to effectively target its resources 
to minimize corporations tax loss. 


Since the Ministry has not assessed the nature and causes of the tax gap, it cannot effectively 
target its efforts to address the tax gap and assess the effect of its efforts in reducing the tax 


gap. 





Recommendation 


To help ensure the achievement of its objective of encouraging the highest 
possible degree of voluntary compliance from taxpayers and thereby reducing 
the tax gap, the Ministry should conduct research into the areas contributing to 
the tax gap and direct the necessary resources to minimize the tax loss. 


Ministry Response 


The Ministry of Finance has an ongoing program of research related to the tax 
gap, and staff at the Ministry have contributed to national publications on this 
subject. Ministry staff closely monitor research undertaken in universities, 
institutes, and other government agencies in order to apply this information to 
Ontario’s situation. The Ministry will increase its research on factors that could 
enhance voluntary compliance and address those factors as opportunities are 
identified. 


Two key determinants of a tax gap—the impact of a changing rate of taxation 
and the level of enforcement activity in a jurisdiction—have been receiving 
concerted attention. 


An additional 170 staff have substantially increased audit coverage, as noted 
elsewhere in the Provincial Auditor’s report. In addition, Ontario’s corporate 
tax rates have fallen dramatically since 1995—the general rate of tax by nearly 
| 20% and the small business tax rate by over 36%. 23 
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MINISTRY OF HEALTH AND LONG-TERM CARE 


3.03—Community Mental 
Health 





BACKGROUND 


In 1976, the Ministry of Health and Long-Term Care began funding community-based 
mental health services to address a growing need for community-based services. Transfer 
payments are provided to community agencies or general hospitals through the Ministry’s 
Integrated Health Care Program to deliver community-based mental health programs and 
to help cover the costs for sessional fees, homes for special care, and other housing with 
supports. [he types of programs and services funded include assertive community treatment 
teams, housing, social rehabilitation, vocational assistance, case management, crisis response, 
day treatment programs, court diversion, clinics, self-help and prevention. These programs 
are primarily designed to meet the needs of the seriously mentally ill who are 16 years and 
over. 


The Ministry of Health Act, the Mental Health Act, the Substitute Decisions Act, the Health 
Care Consent Act and the Homes for Special Care Act all govern certain aspects of the 
community mental health programs funded by the Ministry. In 1999, the Ministry released 
Making It Happen, outlining the Ministry’s three-year strategy for restructuring the mental 
health system “to support much needed changes in the way services are delivered.” 


According to the Ministry, “one of the goals of mental health reform is to create local 
systems of care that will ensure access to a broad range of community-based services and 
support, and provide choices for people with mental illness. These local systems of care will 
enable them to set and realize their personal goals, and acquire the skills and resources 
needed to achieve independence and well being.” 


Making It Happen (1999) identifies people with serious mental illness as the priority for 
community mental health services. According to the Ministry, “the critical dimension is the 
extent of disability and serious risk of harm to themselves or others related to a diagnosable 
disorder.” At the time of our audit, the Ministry estimated that approximately 2.5% of the 
population of Ontario, or 300,000 people, are seriously mentally ill. 


In December 2000, “Brian’s Law” amended the Mental Health Act and the Health Care 


Consent Act. The amendments expand the assessment and committal criteria to include 
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chronically mentally ill persons and allow their families and health professionals to intervene 
at an earlier stage in the committal process. 


During the 2001/02 fiscal year, the Ministry provided approximately $390 million in 


transfer payments for community-based mental health services. 


AUDIT OBJECTIVES AND SCOPE 


The objectives of our audit of community mental health were to assess whether: 


* the Ministry had defined its expectations for community mental health and whether 
adequate procedures were in place to ensure services funded by the Ministry were 
meeting those expectations; and 


* resources were acquired and managed with due regard for economy and efficiency. 


In conducting our audit, we reviewed relevant files and administrative policies and 
procedures and interviewed staff at the Ministry’s head office and three of its seven regional 
offices. We also visited a number of community mental health agencies to gain a better 
understanding of the services being provided and relevant procedures and to corroborate 
information provided to us by the Ministry. 


Our audit was conducted in accordance with the standards for assurance engagements, 
encompassing value for money and compliance, established by the Canadian Institute of 
Chartered Accountants and accordingly included such tests and other procedures as we 
considered necessary in the circumstances. Prior to the commencement of our audit, we 
identified the audit criteria that would be used to address our audit objectives. These criteria 
were discussed with and agreed to by senior ministry management and relate to systems, 
policies, and procedures that the Ministry should have in place. 


Our audit was substantially completed in April 2002. We did not rely on the Ministry’s 
internal audit branch to reduce the extent of our audit work because they had not recently 
conducted any audit work on community mental health services that impacted on our 
audit. 


OVERALL AUDIT CONCLUSIONS 


Since 1988, the Ministry has conducted numerous studies on the future direction of 
Ontario's mental health system. Since 1993, the primary focus has been on the needs of 
people with serious mental illnesses. The Ministry's strategy document, Making It Happen 
(1999), outlines the characteristics of the reformed mental health system, implementation 
priorities, core service requirements and definitions, as well as functional descriptions of 
specific program models. 
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Many of the fundamental issues and concerns identified in our audits over the last 15 years 
have not been comprehensively addressed. In particular, we found that, except in the case of 
assertive community treatment teams, the Ministry still had not clearly defined its 
expectations for community mental health and did not have sufficient information to enable 
it to assess whether mentally ill people were being adequately cared for and whether 

funding provided for community-based mental health services by the Ministry was being 
prudently spent. Specifically, we noted that: 


¢ The Ministry generally did not have standards and performance measures for 
community mental health and had only limited information about whether community 
mental health resources were being utilized efficiently and effectively. 


¢ The Ministry did not have adequate information on the number of people receiving or 
waiting for community mental health services or on the waiting times to access services, 
which limited its ability to assess whether sufficient and appropriate services were 
available to meet the needs of seriously mentally ill individuals. 


e In many areas of the province there is still no comprehensive source of information 
about available mental health services or how to access those services. In addition, there 
is minimal co-ordination among agencies providing services. 


¢ The Ministry had not determined the number or type of housing spaces required to 
meet the needs of seriously mentally ill individuals or whether existing housing was 


meeting the needs of the individuals housed. 


¢ The Ministry had not determined the number and locations of assertive community 
treatment teams needed to provide adequate support services to individuals with serious 
and persistent mental health problems and the level of less intensive services needed to 
meet the needs of individuals who no longer require assertive community treatment. 


With respect to acquiring and managing resources with due regard for economy and 
efficiency, the Ministry had not given sufficient consideration to the funding of community 
mental health agencies based on an assessment of the number of patients requiring services 
and the complexity of patients’ needs. In particular: 


¢ Annual per capita funding in the seven regions of the province ranged from 
approximately $11 to $60; however, the Ministry had not analyzed whether the 
significant variation in per capita funding is resulting in different levels of service for 
individuals with similar needs depending on where in Ontario they live. 


¢ Since 1992, there have been no increases in base funding provided to community 
mental health agencies for programs that were operating at that time. One district 
health council noted that this has forced community mental health agencies “to reduce 
services to the seriously mentally ill in order to stay within existing base budgets.” 


We also concluded that, to provide better accountability to the public and the Legislature, 
the Ministry needed to develop results-oriented performance measures and periodically 
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report publicly on the performance of community-based mental health services in meeting 


the needs of the mentally ill. 


DETAILED AUDIT OBSERVATIONS 
MENTAL HEALTH REFORM 


Mental illness can result in serious disability, hospitalization, suicide, and risks to public 
safety. A 1996 study projected that, in developed countries of the world, the burden of 
mental illness would grow and that depressive illness would become the leading cause of 
disability. 

Over the past 14 years, there have been a number of ministry policy and implementation 
strategies to reform Ontario’s mental health system, including Building Community Support 
for People: A Plan for Mental Health in Ontario in 1988, Putting People First in 1993, and 
Making It Happen in 1999. 


In Putting People First (1993), the Ministry announced a 10-year strategy, commencing in 
1993, for reforming the province’s mental health system, noting that if community services 
were effective in providing care in the community, the hospitalization rate for people with 
mental illnesses should drop. Along with the strategy, the Ministry established measurable 
targets and timelines for the number of beds and for funding. The target for number of 
beds was subsequently modified based on recommendations from the Health Services 
Restructuring Commission in a 1997 discussion paper and in its 1999 Advice to the 
Minister of Health on Building a Community Mental Health System in Ontario. 


Status of Community Mental Health Targets for Spending 
and Number of Beds, 1997-2003 


Status as of Status as of 


Target Area Target for 2003 arch 31, 2002 March 31, 1997 


Ratio of community to 
institutional spending 








Number of hospital 
psychiatric beds per 
100,000 people 


35 
(ultimate target: 30) 





Source of data: Ministry of Health and Long-Term Care 


The Commission noted that “the proposed hospital bed targets are achievable once the 
appropriate Community services and supports are in place to reduce reliance on institutional 
care (especially provincial psychiatric hospitals) and dramatically réduce the need for 
hospital-based treatment services.” The Ministry informed us that it was still planning to 
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meet these targets as part of its “long-standing public strategy to reduce the number of 
beds—only after the establishment of appropriate community-based services.” 


In Making it Happen, which is currently being used to provide implementation direction 
for mental health reform, the Ministry indicates that the key characteristics of a reformed 
mental health system include a continuum of care, in which clients receive services when 
and where they need them. Other characteristics are streamlined access to mental health 
services and the provision of services based on best practices. 


According to the Ministry's implementation plan for Making It Happen, people with 
serious mental illnesses are the priority for mental health services. The implementation plan 
outlines the Ministry’s “strategy to increase the capacity of the system for comprehensive and 
integrated treatment, rehabilitative and support services, while focusing on community 
alternatives wherever possible.” In addition, the plan states that in the year 2002, the 
Ministry is to review and revise implementation strategies and program funding priorities as 
necessary. The Ministry advised us that the evaluation will begin during the 2002/03 fiscal 
year. 


Commencing in May 1999, as recommended by the Health Services Restructuring 
Commission, the Ministry established nine regional Mental Health Implementation Task 
Forces. The task forces were appointed by the then Minister and include representatives 
from psychiatric hospitals, general hospitals, and community mental health agencies, as well 
as general members from communities across the province and consumer and family 
representatives. The task forces, which are expected to complete their work by December 
2002, were asked to look at options to: 


¢ provide a greater range of services in the community; 

¢ improve access to mental health services; 

¢ tailor services to those with mental health needs; 

¢ link services so those with mental illness can move seamlessly within the system; and 
¢ ensure services are based on best practices. 


In addition, in January 2001, a provincial forum comprised of the nine task force chairs was 
established to identify provincial issues that need to be addressed to successfully implement 
mental health reform across Ontario. The provincial forum reports to the Minister. 


Our concern is that a number of the issues currently being addressed, such as the lack of 
community-based support services, were raised by us 15 years ago in our 1987 Annual 
Report and that the mental health best practices that have been identified in the various 


policy and implementation strategies are still to be comprehensively implemented. 
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Recommendation 


The Ministry should ensure that the necessary reforms, including best 
practices identified in the studies, are implemented as soon as possible in 
order to meet the needs of the seriously mentally ill. 


Ministry Response 


The Ontario government is committed to building an integrated system of 
mental health services based on best practices to meet the needs of persons 
with serious menial illnesses. Since 1995, the government has invested more 
than $380 million in mental health care services. 


In order to undertake the government's implementation strategy for mental 
health reform, as outlined in Making It Happen, the government established 
nine regional Mental Health Implementation Task Forces as well as a Provincial 
Forum of Chairs, whose reports are expected to be completed in December 
2002. The Task Forces have been asked to develop regional implementation 
plans to operationalize a restructured local and regional mental health system. 








The Provincial Forum is also preparing provincial level recommendations. 





ACCOUNTABILITY 


Accountability Framework 


According to the Management Board of Cabinet Directive on Transfer Payment 
Accountability, transfer payments should be managed wisely and prudently to achieve value 
for money. The directive lists accountability elements that should be in place, including: 
defined expectations that focus on measurable results; signed agreements that state recipient 
reporting requirements and bind recipients to achieve specific, measurable results; and 
ministry monitoring and corrective action in cases of recipient non-compliance with the 
agreement. 


In our 1997 Annual Report, we noted that the Ministry had recognized the need to 
strengthen its accountability relationship with community mental health agencies and had 
drafted a service agreement that required agencies to use ministry funding in accordance 
with approved operating plans and budgets. At that time, we were advised that these 
agreements were being put in place. During our current audit, we found that agreements 
with some assertive community treatment teams (discussed later in this report) had been 
signed, but the Ministry generally still did not have agreements signed with community 
mental health agencies. At the completion of our audit, transfer-payment agreements and 
operating manuals were being distributed to all mental health and addiction agencies. The 
Ministry indicated that it expects agreements to be finalized in fali 2002 and believes that 
this will go a long way in strengthening the accountability framework. 
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In Making It Happen, the Ministry states that it is “committed to the principle of greater 
accountability in the reformed mental health system. The mental health system will be 
measured against the accountability framework that is to be developed.” We were advised 
that developing a mental health accountability framework is one of the priority areas of the 
Mental Health Implementation Task Forces and that a Mental Health Accountability 
Framework Reference Group, comprising ministry and external representatives, was 
established in the latter part of the 2001/02 fiscal year to provide advice to the Ministry on 
developing such a framework. Ministry staff also indicated that they were proceeding with 
the development of a provincial accountability framework for mental health services. In that 
regard, the Reference Group had prepared a summary of accountability approaches and 


strategies in other jurisdictions. 


Recommendation 
To better hold community mental health agencies accountable for the services 


provided and for the prudent management of the funds they receive, the 
Ministry should ensure that all basic elements of the Management Board of 
Cabinet Directive on Transfer Payment Accountability are addressed, including 
signed agreements that require recipients to achieve specific, measurable 
results. 





Ministry Response 


The Ministry has now developed a mental health accountability strategy that 
consists of four elements: 








¢ anaccountability framework that outlines goals, purposes, performance 
domains, and indicators: in addition, performance measures based on the 
domains and indicators, as well as data collection tools, are being 
developed by the Ministry’s Health Care Programs Division; 

¢ legal agreements between the Ministry and transfer-payment recipient 
agencies; 

¢ arevised operating manual for mental health and addiction agencies; and 

e¢ hospital accountability mechanisms. 





A service agreement has now been developed and sent to all community 
mental health agencies for signature. An initial program manual has also been 
issued. A working group is in place to develop the basic framework for 
specifying program expectations (that is, service units, client groups, and the 
key indicators to measure performance). This will evolve into schedules for the 
service agreements. 
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Performance Measurement and Reporting 


Effective accountability requires that program clients, their families, the Legislature, and the 
general public be provided with timely, reliable information about the performance of 
community mental health programs. Performance information is needed to enable the 
Ministry to evaluate the impact of mental health reform, including the impact of changes in 
the structure and the organization of services. For example, performance information is 
needed to measure the extent to which mental health reform is improving the quality of life 
of community mental health clients and their families. 


In our 1997 Annual Report, we recommended that the Ministry establish performance 
benchmarks and outcome measures and monitor community mental health programs 
against them. This would help ensure that agreed-upon community-based services are being 
provided and that funding is reasonable and consistent. At that time, the Ministry indicated 
that mental health benchmarks, targets, and outcome measures were being developed and 
that the efficiency and effectiveness of service delivery would be the focus of further 
outcome measures that would span both the hospital and community sectors. 


During our current audit, we found that, overall, the Ministry had limited information 
about whether community mental health resources were used efficiently and effectively and 
that the Ministry was still identifying performance indicators for mental health services. In 
addition, while performance measures for employment services and supports had been 
developed, they had not yet been fully implemented. As the Ministry noted in the terms of 
reference of the Mental Health Accountability Reference Group, various provincial data 
collection tools, evaluation and monitoring mechanisms, and performance indicators have 
been developed over the past several years. However, these are not being used consistently. 


Many other jurisdictions are developing and implementing performance measures for 
mental health that address performance at system-wide, program, and client levels. Key 
measures assess Outcomes such as improvement in clients’ mental health status, clients’ ability 
to function in society, and client satisfaction. Although outcomes may be affected by factors 
unrelated to an agency’s services—for example, the severity of an individual’s mental 
illness—outcome information can assist in improving program quality and the Ministry’s 
ability to assess the cost effectiveness of programs. 


We found that many jurisdictions in North America were already reporting publicly on the 
performance and outcomes of their community mental health systems. However, Ontario 
had not issued public performance reports for community mental health that included 
information on the Ministry’s progress in meeting mental health reform targets. 









Recommendation 


To help achieve ongoing improvements in providing community mental health 
services, the Ministry should: 
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¢ develop and implement appropriate performance measures that objectively 
m 


easure the success of agencies in meeting the needs of the seriously 
mentally ill; 


¢ regularly report publicly on performance, including reporting on the impact 
of mental health reform; and 


e take corrective action where required. 








Ministry Response 


The Integrated Health and Planning Division has now developed an 
accountability framework identifying key domains, and the Mental Health 
Accountability Reference Group will seek to identify indicators on which to 
report performance in those domains. The service agreement specifies 
reporting types and the timetable. Regional offices use a risk-based monitoring 
process to determine when and what corrective action is required. 


Resources for hardware and software will be required for the planned 
information-system design. | 








Monitoring Community Mental Health Agencies 


At the time of our current audit, there were approximately 370 community mental health 
agencies receiving ministry funding to provide mental health services and supports such as 
assertive community treatment, case management, crisis intervention, housing, consumer 
and family self-help, and vocational rehabilitation. 


Community mental health agencies must submit annual operating plans, including 
program budgets, to their regional ministry offices and local district health councils. District 
health councils review the operating plans and provide comments to the Ministry if there 
are significant program changes. The review and approval of operating plans by the 
Ministry is to be completed by early June of each year, taking into consideration any 
comments from district health councils. 


For the 2001/02 fiscal year, each community mental health agency’s annual operating plan 
was required to include: a proposed budget with explanations of any significant changes 
from the prior year; program and client goals and objectives for the upcoming year; and a 
report on the agency’s achievement of the goals and objectives established in the prior year’s 
operating plan. 


We found that the operating plans for the 2001/02 fiscal year were generally recetved and 
approved by the Ministry within a reasonable time frame. All three regions we visited used 
checklists to ensure that information was submitted in accordance with ministry 
requirements. However, since the process was done manually rather than electronically, no 
ageregate data on the agencies was available. 
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We reviewed a sample of 2001/02 operating plans at the three regions we visited and noted 
that they generally included a budget with explanations of any significant changes from the 
prior year, client goals and objectives for the upcoming year, and the agency’s achievement 
of goals and objectives related to the prior year’s operating plan. However, we also noted 
that, while agencies are required to report in their annual operating plans the number of 
individuals served by each type of service, they are not required to report the total number 
of individuals served or the number of seriously mentally ill individuals served. Without such 
information, the Ministry is hampered in its ability to make planning and funding decisions. 


Prior to 1996 the Ministry conducted detailed reviews of community mental health 
agencies, in which it assessed basic aspects of program organization and delivery, including 
whether appropriate individuals received services and whether treatment goals for 
individuals were established and monitored. Since 1996, the Ministry has monitored 
agencies primarily through reviews of operating plans, informal visits to agencies, and 
telephone contacts. However, this process does not provide the information on the 
individuals served nor the assurance of the quality of the services provided that would be 
obtained from a detailed review. 


Program standards establish performance expectations. For example, quality improvement 
standards would enable the Ministry to evaluate an agency’s procedures for assessing and 
improving the quality of services it provides. We noted that, except for assertive community 
treatment teams (which are covered later in this report), the Ministry had not established 
standards for community mental health programs. Other jurisdictions in North America 
have established standards for community mental health programs such as supported 
housing and employment services. 


A properly designed information system that includes the data required for appropriate 
performance measures is essential for the Ministry to identify agencies that require further 
review and would enable the Ministry to utilize its resources more efficiently. 





Recommendation 


To help ensure that resources are utilized efficiently and are achieving their 
intended results, the Ministry should: 


* ensure that it has adequate information to make planning and funding 
decisions; and 

e require that agencies submit information on the number of seriously 
mentally ill individuals who received their services. 


To help ensure that community mental health agencies provide high quality 
programs, the Ministry should: 


* establish standards against which programs can be evaluated; and 


e implement agency reviews focusing on those agencies identified as high 


risk. 
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Ministry Response 


Conversion of the existing Community Mental Health (CMH) Budget & 
Inventory System to a CMH MIS (Management Information System) chart of 
accounts in the Ontario Hospital Reporting System will provide the Ministry 
with more details for in-year reporting using standard financial and statistical 
accounts and consistent definitions of mental health services. This will allow 
the comparison of expenditures across and within the various mental health 
sectors. 








Data collected by the Common Data Set—Mental Health, which employs the 
same set of mental health services that are defined in the CMH MIS chart of 
accounts, will capture additional clinical details to allow the Ministry to have 
information regarding services provided by the community mental health 
agencies to seriously mentally ill individuals. 


The Mental Health Accountability Framework includes performance domains 
and indicators, which are based on the goals and principles set out in Making It 
Happen. Performance measures for each indicator are under development. 
Standards and benchmarks, based on best practices, are also under 
development. : 


Ministry regional offices will continue to undertake or contract agency reviews 
in response to program or financial concerns. 








ACCESS TO COMMUNITY MENTAL 
HEALTH SERVICES 


In Making It Happen, the Ministry states that “access to mental health services in Ontario 
can be confusing and time-consuming for clients and their families/key supports.” In that 
regard, we noted that in many areas of the province there is no central source of 
information about available mental health services or how to access these services. According 
to Making It Happen, in some areas of the province there is minimal co-ordination among 
agencies that provide similar or identical services, and individuals “are often unclear as to 
which services are suitable to their needs and how to access them. As a result, they may seek 
several services at once (undergoing separate assessments for each service) and they may be 
on several waiting lists at the same time.” 


To reduce duplication, the Ministry funded a project to co-ordinate access to housing in 
Toronto. In March 2002, the “Report on the Planning and Development of a Coordinated 
Access System for Mental Health Supportive Housing in Toronto” was submitted to the 
Ministry, proposing a single housing application form. At the time of our audit, a budget 
detailing the cost of implementing this system was still to be prepared by the agencies 
involved with the project. The Ministry is also funding a project to develop a co-ordinated 


housing access system in another area of the province. 
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Community mental health agencies generally have their own access and exclusion criteria. 
For example, to access services, we found that some agencies required individuals to be 
diagnosed with a serious mental illness while other agencies did not. This creates a risk that 
some agencies may exclude individuals requiring more services. In addition, because 
agencies generally have their own intake and assessment processes, individuals apply 
separately to each agency to access services. This results in an inefficient use of agency staff 
and can be frustrating to individuals seeking assistance. 


In February 2002, the Ministry issued “A Guide to Developing Recommendations on 
Streamlining Access to Mental Health Services and Supports.” According to the Ministry, 
this guide is intended to assist the Mental Health Implementation Task Forces, regional 
offices, and stakeholders to provide optimum services to clients and to minimize the 
duplication of services and maximize efficiency. 


We were advised that the Ministry had no information available on the overall number of 
seriously mentally ill people waiting for services or the overall waiting times to access services 
provided by community mental health agencies. We also noted that agencies generally did 
not record waiting lists in a consistent manner, and waiting lists were not co-ordinated 
among different agencies. For example, some agencies limited the number of people that 
could be on their waiting lists, and others did not. Information on waiting lists and waiting 
times is needed by the Ministry to help determine the need for specific types of services. 


Information about the mental health of people living in different regions of Ontario and 
the prevalence of specific mental disorders in those regions is important for helping to 
determine the need for services and for assessing the effectiveness of services provided. For 
example, the Ministry would be better able to identify and address service gaps if it had 
information about the number of seriously mentally ill individuals in a region. The 
information would assist in the effective planning, evaluation, and funding of community 
mental health services. 


Over the past two years, community comprehensive-assessment projects were undertaken 
throughout the province to identify the care needs of clients in community mental health 
programs. These projects compared client needs with the care being provided. 


The project reports that we reviewed stated that the needs of individuals were often either 
not sufficiently met or more than sufficiently met. One report noted that its results 
reinforced the importance of clearly defining program admission criteria, conducting 
ongoing utilization reviews to ensure that changing levels of need are identified and 
addressed, and reviewing the complement of program types required to meet client needs. 
However, the projects did not recommend the level of services needed to meet client needs 
and did not assess the needs of persons with mental health problems who currently were not 
using the mental health system. We were advised that the Ministry was in the process of 
analyzing the results of the projects. 


The most recent data that the Ministry has on the prevalence of mental disorders is from the 
1990/91 Mental Health Supplement to the Ontario Health Survey. While this survey 


eee 
98 2002 Annual Report of the Provincial Auditor of Ontario 





provided valuable information, only limited information was available regarding the 
seriously mentally ill and the severity and level of disability caused by some mental disorders. 
It also did not provide information concerning individuals who may have significant needs 
for services, such as individuals with psychotic disorders, including schizophrenia. Ministry 
staff informed us that a survey similar to the 1991 Ontario Health Survey was being 
conducted and that the results were expected in 2003. 


In Making It Happen, the Ministry stated that “Clients, families/key supports and service 
providers will be able to contact a central source to get information about mental health 
services and how to access them.” In 2001, the Ministry provided funding to the Ontario 
Drug and Alcohol Registry of Treatment (DART) to develop a plan for a provincial registry 
of mental health services and an information line. DART is currently “a province-wide 
information and referral service available to service providers and members of the general 
public, including substance abusers and family/friends of substance abusers.” In May 2002, 
DART submitted a proposal to the Ministry for funding to develop and implement the 
mental health services registry. According to the proposal, the registry would support the 
development of two mechanisms identified in Making It Happen (1999): streamlined access 
for clients and families; and service/system accountability. At the conclusion of our audit 
fieldwork, the Ministry had not yet made a decision on whether to proceed with this 
initiative. 








Recommendation 


To help ensure timely and equitable access to services, the Ministry should: 


e review the feasibility of further co-ordinating access to services, including 
establishing common intake and assessment criteria; 


e obtain and analyze overall waiting lists and waiting times to help determine 
the need for specific types of services; and 


e ensure that public information on community mental health services and 
how to access those services is readily available. 


Ministry Response 


In February 2002, the Ministry released the document “A Guide to Developing 
Recommendations on Streamlining Access to Mental Health Services and 
Supports.” This document sets out four key features for streamlining access to 
mental health services, along with goals, key considerations, and required 
outcomes for each feature. The four features are: centralized information and 
referral functions; facilitating access to consultation services provided by 
psychiatrists; minimizing the number of assessments; and fewer points of 
entry to mental health services. The Mental Health Implementation Task Forces 
and other stakeholders are to use the guide to develop recommendations for 
streamlining access to mental health services. The Task Force’s final 
recommendations are expected in December 2002. 
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At the community mental health agency level, waiting lists should be client 
specific and shared among agencies. The Ministry is proposing an initiative for 
a client linkage system that will provide community mental health agencies 
with the ability to share waiting lists. 


At the ministry level, waiting-time statistics should be aggregated to provide 
data on service type and volume demand so that waiting lists can be 
strategically managed. This data element will be added to the Community 
Mental Health Management Information System proposal. 


The Ministry has supported the planning for the development of a provincial 
mental health information system, building upon the existing Drug and Alcohol 
Registry of Treatment. Steps are being taken to seek funds for the 

establishment of this information network. 
Deena 








INFORMATION SYSTEMS 


To assist them in co-ordinating and improving care and services, community mental health 
agencies need up-to-date and accurate information about their clients. Such information 
enables an agency to respond promptly and appropriately to individual client needs. The 
Ministry also needs relevant information to effectively monitor the performance of agency 
programs and the mental health system as a whole and to assess the impact of mental health 
reform. 


People with serious mental illnesses often need access to a broad range of services, such as 
assessment, treatment, housing with supports, and vocational assistance. Services are often 
provided by different local community mental health agencies, which may result in 
duplication of consumer information and a lack of co-ordination among local services. 
Individuals whose mental illness affects memory and thought processes are at a greater risk 
of losing or failing to make contact with the appropriate agencies. Co-ordination of mental 
health services helps ensure that people receive the services they need. 


We noted that other jurisdictions had electronic client records that integrated information 
from a number of sources. For example, the National Health Service in England is 
implementing an integrated electronic mental health record. In addition, other jurisdictions 
are developing information systems to address the information needs of multiple users, 
including community agencies, government, and mentally ill individuals and their families. 
Confidentiality and security of mental health information is also addressed as part of these 
systems. 


A key aspect of such systems is a minimum data set, which is a uniform set of data that 
enables the collection and exchange of information among different service providers. It 
provides information such as: the type and volume of services provided, who provides the 
services, who receives the services, and service outcomes. Specific setvice definitions that 
establish uniform descriptions of the services provided by community mental health 
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agencies are needed to develop a minimum data set, and such definitions enable meaningful 
comparisons to be made of the costs and outcomes of services among programs. 


As far back as our 1987 Annual Report, we reported that the Ministry lacked the 
information necessary to plan and assess the operations of community mental health 
programs. In response to a related recommendation in our 1997 Annual Report, the 
Ministry indicated it would be developing a mental health minimum data set. During our 
current audit and 15 years after we first raised the issue, we found that Ontario still has no 
integrated client information system. We also found that there are no system-wide 
agreements concerning who may access client information or under what circumstances. In 
many instances information is still largely paper-based. Except in the case of assertive 
community treatment teams, which are discussed later in this report, the Ministry has not 
yet developed service definitions that would enable comparisons among programs. In 
addition, we were informed that, while the Ministry has developed a minimum data set, its 
implementation has been delayed. 


Many seriously mentally ill individuals receive care from various health care organizations as 
well as community mental health agencies. In order to co-ordinate information from all 
these service providers, many jurisdictions use unique client identifiers as part of their mental 
health information systems. The unique client identifier enables agencies to access key client 
data while safeguarding privacy and confidentiality. A unique client identifier also enables 
an individual’s records in one information system to be matched with those in another 
system for purposes of evaluation and planning. For example, information on individuals 
receiving services from mental health agencies who also receive other medical services or 
who are involved with the justice system would assist the Ministry in determining the 
effectiveness of mental health programs and the need for programs. 


At the time of our audit, the Ministry was in the process of establishing performance 
measures for mental health services. An effective mental health information system would 
incorporate the information required for these performance measures and, at the 
community level, provide data to help minimize duplication of services and maximize 


efficiency. 





Recommendation 








To better support the provision and co-ordination of community mental health 
services, the Ministry should design, implement, and appropriately utilize a 
mental health information system that captures relevant service and client 
data. 


Ministry Response 


The development of the Common Data Set—Mental Health is nearing 
completion. Following consultation and implementation, the Ministry will have 
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basic but consistent data on the definition of services, the number of clients, 

and the types and amounts of services received by mental health clients. | 
The Ministry is also developing a proposal for a client linkage system. The 
system's aim is to ensure that community mental health services are integrated 
and co-ordinated in such a way that only the most appropriate services are 


provided, in the interests of clients’ well-being and community mental health 
resource utilization. 
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HOUSING 


In the various mental health reform initiatives that have been proposed by the Ministry, 
there is general agreement that, with appropriate support, most seriously mentally ill 
individuals can live in the community. Research indicates that the housing needs and 
preferences of people with severe mental illness vary considerably, and therefore varying 
types of housing alternatives are needed to meet the needs of these individuals. Housing 
options funded by the Ministry include: 


* housing under the Mental Health Homelessness Initiative: 
* supportive housing; 

° homes for special care; 

* approved homes; and 


e Habitat Services. 


Housing Needs 
In Making It Happen (1999), the Ministry promotes establishing a comprehensive mental 


health housing framework. The framework includes increasing the availability and use of 
accommodation that is desired by seriously mentally ill individuals and providing support 
services tailored to these individuals’ needs. 


In our 1987 Annual Report, we commented on the shortage of good-quality affordable 
housing for individuals discharged from psychiatric hospitals and noted that this was a 
major cause of relapse and readmission into psychiatric hospitals. During this audit, we 
found that the Ministry still had not determined the number or types of housing spaces 
required to address the needs of seriously mentally ill individuals. To complicate matters, the 
Ministry did not have any current province-wide information on the number of seriously 
mentally ill individuals who were homeless or inadequately housed. However, in 2001, the 
Ministry asked district health councils to conduct local research on the housing situation of 
persons with serious mental illnesses. The Ministry indicated that the resulting reports were 
designed to provide factual data but not specific recommendations. Nevertheless, the 
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Ministry believes this research may provide additional data on the need for housing support 
for the seriously mentally ill. 


According to a 2001 Canadian Mortgage and Housing Corporation survey, three major 
urban centers in Ontario had rental accommodation vacancy rates of less than 1.5%. In 
addition, some areas of the province had very long waiting lists for social housing. Ministry 
staff also believed there were long waiting lists for ministry-funded supportive housing and 
advised us that one consequence of the housing shortage was that some seriously mentally ill 
individuals may be living in poor-quality housing or in a highly structured setting not 
appropriate for the individual. 


Homes for Persons with Special Needs is an initiative introduced by the government to 
consolidate the Ministry of Health and Long-Term Care’s homes for special care, approved 
homes, and Habitat Services with the Ministry of Community and Social Services’ 
domiciliary hostels. The stated objective of this initiative is to improve the delivery of care for 
people with similar special needs by establishing common levels of funding and consistent 
standards, including those for staffing, drug control systems, nutrition, cleanliness, and 
assistance with aspects of care. While such an initiative is encouraging, it has been 15 years 
since we first raised a concern about the housing needs of the seriously mentally ill. 



























Recommendation 


To help address the long-standing problem of affordable and appropriate 
housing for the seriously mentally ill, the Ministry should: 


e assess the number and types of housing units needed in different areas of 
the province and whether ministry-funded housing is meeting the needs of 
individuals already housed; and 


e take appropriate steps to address the assessed housing needs. 


Ministry Response 


In 2001, district health councils (DHC), on behalf of the Ministry, conducted 
local research on the housing situations of persons with serious mental 
illnesses. The DHCs consulted with key stakeholders, particularly consumers 
of mental health services, their families, mental health agencies, and housing 
providers. The consultations assessed current mental health housing and 
related support service delivery. 


The survey also included suggestions from stakeholders on how to improve 
housing support for persons with menial illnesses. The study results were 
provided to the Mental Health Implementation Task Forces to help them in 
developing local mental health housing plans. 





Based on the Task Forces’ recommended local mental health housing plans, 
the Ministry will put forward appropriate housing strategies as part of the 
business planning cycle. 
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Outcome measures relating to “housing meeting required needs” can be 
achieved through the psychosocial rehabilitation tool kit when fully 
implemented through an information system. 












Mental Health Homelessness Initiative 


The 1998 Report of the Provincial Task Force on Homelessness noted that perhaps as many as 
one-third of all homeless people suffer from a serious mental illness. In March 1999, the 
Ministry announced a provincial homelessness strategy that included providing housing and 
support services for people with serious mental illnesses. Phase I of the Mental Health 
Homelessness Initiative was to develop approximately 1,000 new housing spaces with 
support services for people with serious mental illnesses in Toronto, Hamilton, and Ottawa. 
Phase II of the initiative, which was announced in November 2000, was to provide at least 
2,600 additional supportive housing units throughout the province for seriously mentally ill 
individuals who were homeless or at high risk of homelessness. Phase IT units were to be 
created over a two-year period. 


For both phases, non-profit agencies received funding from the Ministry to acquire the 
needed housing units by entering into leases with landlords and then subletting the units to 
individuals in the target group. However, where vacancy rates were low and affordable 
rental units were lacking, the Ministry provided funding to the agencies to purchase 
properties and, if necessary, to renovate them. Based on ministry information, by March 
2002, 950 units had been secured through Phase I of the initiative and about 970 units 
had been secured through Phase II, of which about 80% were rental units and 20% had 
been purchased. 


We were advised that transfer payment agreements will be used to formally establish the 
accountability relationship between the Ministry and the agencies providing the supports. 


For leased units, the Ministry sets the rate paid by tenants and pays the agency the 
difference between what the tenant pays and the lease rate; it also receives occupancy 
information from the agencies. On the other hand, for properties that were purchased, at 
the time of our audit, agencies determined the amount they charged tenants. The rent 
charged to tenants is intended to cover the operating costs of the property. The Ministry 
does not formally require that agencies with purchased properties provide information 
about the number of units occupied by seriously mentally ill individuals. We also found no 
evidence that the Ministry reviews the rents charged. 


We reviewed a sample of properties purchased under Phase II and found that, when 
purchased, some properties were already rented to tenants whose occupancy is protected 
under the Tenant Protection Act. Ministry management stated that these tenants would leave 
over time and that the accommodation would then become available to house the seriously 
mentally ill. In the interim, the agencies collect rent from the existing tenants. Thus, 
although the Ministry has provided funding to agencies to buy these properties, it does not 
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know when they will actually house seriously mentally ill individuals. At our request, 
ministry staff obtained information about Phase II purchased properties and found that 
about 90 units still had the existing tenants and accordingly were not available to provide 
housing for the seriously mentally ill. 


‘To determine whether Phase I of the Mental Health Homelessness Initiative had met its 
objectives, the Ministry surveyed Phase I housing providers for the period from April 1, 
2000 to March 31, 2001. The survey results provided the Ministry with a variety of data, 
including a profile of the individuals housed. The profile indicated that these individuals 
were seriously mentally ill and either homeless or at risk of being homeless. However, the 
Ministry did not verify if the information was accurate. The Phase I agencies we visited 
confirmed that individuals housed in facilities funded through Phase I of the homelessness 
initiative were homeless or at risk of being homeless but they stated that a formal diagnosis of 
a serious mental illness was not required when assessing eligibility for housing as it was not 
available at the time of the assessment. 


The Ministry is also funding a two-year evaluation of Phase I to assess the achievement of 
certain outcomes for the individuals housed and the impact the initiative has had on the use 
of emergency shelters, hospitals, and forensic services. A final report is expected by 

March 31, 2003. 














Recommendation 


To help ensure that the Mental Health Homelessness Initiative is meeting its 
objectives of providing housing with supports to seriously mentally ill 
individuals, the Ministry should: 


e establish a formal process to obtain information about occupancy in 
housing purchased with ministry assistance; 
e establish accountability agreements with all agencies; and 


e ensure that funding is only provided for properties that are able to provide 
housing and support services for people with serious mental illnesses. 





Ministry Response 


The Ministry now requires agencies that have purchased properties to 
complete a year-end form that collects unit activity (that is, occupancy/ 
vacancy) on a monthly basis. 


Security agreements that outline accountability as it relates to the purchased 
property will be signed for all purchased properties. 





Service agreements relating to the support services provided in the purchased 
properties under this initiative are in the process of being signed with the 
agencies. 
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All properties will be providing housing and support services for persons with 
serious mental illnesses. In cases of tenanted buildings, this will occur over 
time. | 









Supportive Housing 


Supportive housing is non-profit, subsidized housing that includes support services such as 


case management, social rehabilitation, assertive community treatment, and, to some extent, 
crisis intervention. Effective April 1, 1999, the province transferred responsibility for 
approximately 3,100 supportive housing units from the Ministry of Municipal Affairs and 
Housing to the Ministry of Health and Long-Term Care. For the 2001/02 fiscal year, 
expenditures for these housing units were budgeted at $26.5 million. About 75% of the 
housing units were for mentally ill individuals, while the rest serve long-term care or 
substance-abuse clients. 


Agencies responsible for supportive housing units were generally required to have operating 
agreements with the Ministry detailing the respective responsibilities of the agencies and the 
Ministry. Ministry management informed us that, while there were no operating 
agreements covering some supportive housing projects, future planning for the entire 
supportive housing portfolio would include a review of operating agreements, and potential 
reform and consolidation of housing programs. 


Since 1993, the main priority for mental health services has been the seriously mentally ill. 
However, at the time of our audit, the Ministry did not have any system-wide information 
about the individuals being housed in supportive housing units, or how many units were 
allocated to the seriously mentally ill. Without this information, the Ministry is unable to 


assess the extent current housing actually targets and serves the seriously mentally ill. 


We selected a sample of agreements covering over 500 supportive housing units and found 
that only about 10% of the units actually stated that they were specifically dedicated to the 
seriously mentally ill. Many units were dedicated to individuals with “special needs,” 
including people with serious mental illnesses and also individuals who have needs that are 
not associated with mental illness. Ministry staff explained that most of the supportive 
housing units were created prior to the time when seriously mentally ill individuals were 
identified as the priority group for community mental health services. However, there is no 
requirement that, when units become vacant, first priority be given to the seriously mentally 


ill. 


Prior to the transfer of the housing units to the Ministry of Health and Long-Term Care, 
the Ministry of Municipal Affairs and Housing conducted operational reviews to monitor 
the housing providers’ compliance with the financial and operating requirements of the 
operating agreements. This included assessing whether housing was being provided to the 
appropriate individuals. We were informed that the Ministry had begun conducting 
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reviews in the summer of 2002. Since support services are an integral part of supportive 
housing, it would be beneficial to review, at the same time, support services being provided. 





Recommendation 


To help ensure that supportive housing serves individuals who are seriously 
mentally ill and to assist in assessing the need for additional housing, the 
Ministry should: 


e determine the extent to which existing housing is actually targeting and 
serving individuals who are seriously mentally ill; and 


e ensure that first priority is given to the seriously mentally ill. 


Ministry Response 


The Ministry will be looking at the targeting of the client group and plans to 
include the client group as part of the operational review process in mental 
health projects. The regional offices will include this as part of their review of 
the annual budgets. 


The development of the Common Data Set—Mental Health (CDS—MH) is 
nearing completion. CDS—MH will provide baseline and current data that will 
capture mental health clients’ living arrangements and types of residence. 
These data will allow the determination of whether supportive housing is 
targeted and provided to the seriously mentally ill, in order that the seriously 
mentally ill can be given the highest priority. 











Homes for Special Care Program 
The Homes for Special Care Program, which was established in 1964 under the Homes for 


Special Care Act, provides accommodation in private residences with 24-hour supervision 
and assistance with activities of daily living. As of March 2002, there were 157 homes that, 
during the 2001/02 fiscal year, served approximately 1,800 individuals at an estimated cost 
of $28.6 million. Provincially run psychiatric hospitals and psychiatric hospitals that have 
been divested by the province are responsible for placing individuals in homes for special 
Gare, 


The Homes for Special Care Act and regulations require that homes for special care are 
licensed annually. Inspections are conducted by staff from the psychiatric hospitals that 
place individuals in the homes, in conjunction with inspections by the local fire department 
and public health unit. All reports are forwarded to the Ministry. We reviewed a sample of 
files at the Ministry and noted that annual inspections of homes for special care were often 
not received by the Ministry on a timely basis. 
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Although the Homes for Special Care Act does not stipulate any standards for the quality of 
resident care, July 2001 updated interim operating guidelines issued by the Ministry set out 
the minimum standards of care along with specific indicators to be used in the assessment 
and monitoring of those standards. We were advised that regional training on these 
guidelines has been provided. In our 1997 Annual Report, we noted that while homes for 
special care were inspected for adherence to the guidelines in place, compliance was not a 
requirement for licence renewal, and this was still the case at the time of our current audit. 





Recommendation 


To ensure that Homes for Special Care provide appropriate and consistent 
resident care across the province, the Ministry should ensure that: 


e inspections of the homes are completed and followed up on and 
deficiencies are corrected on a timely basis; and 


e adherence by the homes to minimum standards of care is a condition for 
licence renewal. 


Ministry Response 


Annual inspections of the Homes for Special Care are completed in the fall of 
each year. However, co-ordination with the required public health and fire 
inspections is not within program control. Licences are not issued without all 
inspections being completed. A large number of the homes are located in rural 
areas, SO annual inspections are done by volunteer fire departments, and 
timing is often problematic. Also, some municipal fire departments feel these 
inspections are not part of their role and are Starting to balk at doing them, 
particularly when asked to do them within the program timetable. 


Residential Home Reports are completed by field staff and accompany the fire 
and public health reports as part of the annual re-licensing process. Following 
the regionalization of the program and the hiring of regional co-ordinators, 
these Residential Home Reports are now being monitored closely. When 
needed, licences are granted with written notification of areas of concern that, 
if not rectified, may result in the Minister exercising other options, such as non- 
renewal of a licence. These homes are monitored while non-compliance issues 
are being resolved. Outstanding issues relating to non-compliant homes are 
being tracked to their necessary resolution, and actions are being taken to 
cease business with those unwilling to meet the established goals. 











EARLY INTERVENTION PROGRAMS 


Early intervention programs offer strategies designed to limit the duration of a psychosis and 
prevent relapse for people experiencing their first episode of psychotic illness. The strategies 
make use of antipsychotic medication, psychosocial rehabilitation, and family support and 
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education. The “Best Practice Checklist”, part of the document entitled Accountability and 
Performance Indicators of Mental Health Services and Supports prepared by the Federal/ 
Provincial/Territorial Advisory Network on Mental Health, recommends the addition of 


early intervention programs within the continuum of care programs in a reformed mental 


health system. 


In 1999, the Ontario Working Group on Early Intervention in Psychosis was formed to 
develop an effective treatment and support system for the early stages of psychosis. On 
January 31, 2001, the working group submitted a comprehensive proposal for the first 
phase of its provincial strategy for early intervention in psychosis. At the time of our audit, 
the proposal was being considered in the context of work being done by the Mental Health 


Implementation Task Forces. 


In addition, in the 2001/02 fiscal year, the Ministry provided funding to the Ontario 
Mental Health Foundation to undertake a two-year descriptive study of individuals who are 
experiencing the onset of first-episode psychosis. The study is being conducted at four early- 
intervention treatment programs operating in Ontario. Results of the study are expected in 


Spring 2003. 


We will follow up on the Ministry’s early intervention activities during our next audit of 


community mental health. 


ASSERTIVE COMMUNITY TREATMENT 


Assertive community treatment (ACT) was developed approximately 30 years ago by 
health-care professionals concerned about the continual re-admission of psychiatric patients 
to psychiatric hospitals. ACT is designed to provide care in the community for individuals 
with serious and persistent mental health problems, including treatment, rehabilitation, and 
support services. A multi-disciplinary team provides care 24 hours a day, seven days a week. 
Contacts with mentally ill individuals served by the team are designed to be flexible and 
available for as long as needed by the individuals. 


ACT teams usually comprise 10 to 12 full-time clinical staff serving individuals with severe 
symptoms and impairments, some of whom may be hesitant to access community mental 
health services or may otherwise be hospitalized. Accordingly, staff-client ratios are low. 


The first Ontario ACT team was formed in 1990. In 1998, the Ministry began to 
implement ACT across the province. As of March 2002, there were about 60 ACT teams 
in Ontario, of which approximately 45 were funded through community mental health. 
The remainder are funded through hospital budgets. In the 2001/02 fiscal year, 
expenditures for ACT teams funded through community mental health totalled 
approximately $45 million. 
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Need for Assertive Community Treatment 


Various studies indicate that ACT is an effective way of caring for seriously mentally ill 
people in the community, with benefits that include improved retention in treatment 
programs and greater housing stability. Research also indicates that when ACT is targeted at 
high users of hospital inpatient care, it may result in savings. However, since ACT is a high- 
cost program, it is critical that it be correctly targeted to individuals requiring ACT services 
and not include those who would do equally well in less intensive, lower-cost programs. 


The Ministry has not determined the total number of teams required or the number of 
teams required in different regions of the province. The required number and distribution 
of teams is affected by various factors, including the availability of less intensive, lower-cost 
services. The Ministry indicated that it is awaiting the recommendations of the Mental 
Health Implementation Task Forces, which may impact on the assertive community 
treatment program and other services. 


Research indicates that, over time, some ACT clients can be well served by less intensive and 
less costly services with no negative effects. The Ministry’s standards for ACT teams include 
criteria for discharging clients based on improved functioning and a consistent pattern of 
decreased need for ACT services. However, the Ministry has not determined how less 
intensive services for clients discharged from ACT should be structured. Other jurisdictions 
have implemented a multi-tiered system of caring for seriously mentally ill clients that 
includes less intensive services. 


Monitoring Assertive Community Treatment Teams 


In 1998, the Ministry adopted standards for its ACT teams. These standards deal with the 
structure and functions of the teams and admission and discharge criteria. The Ministry 
included mandatory adherence to the standards in service agreements with ACT teams and 
initially planned to have most established teams sign such agreements by March 31, 1999. 
However, in the three regions we visited, we found that most ACT teams had not yet signed 
agreements. Ministry staff indicated that they planned to have service agreements in place 
for most ACT teams by early in the 2002/03 fiscal year. 


The Ministry monitors the performance of ACT teams through the review of operating 
plans and through a process similar to accreditation, whereby ACT teams are assessed for 
compliance with Ontario’s ACT standards and adherence to expert consensus on ACT best 
practices. 


However, the Ministry has no computerized information system that routinely collects 
information from ACT teams. In March 2001, the Ministry conducted a pilot survey on 
how many individuals ACT teams served and whether the teams were serving the target 
population. Although the Ministry had some concerns about the quality of the data 
obtained, the survey results suggested that ACT teams were serving the intended 
population. The survey also indicated that the average regional caseload varied from 17 to 
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32 clients per team, with an overall provincial average of 30 clients. Generally, a full 
caseload consists of 80 to 100 clients per team. 


Ministry staff informed us that many teams were still relatively new at the time of the survey, 
and it was expected that a spring 2002 survey would indicate that most teams were closer to 
their full caseload. The Ministry indicated that it expects teams to reach full capacity in two 
to three years and that it is awaiting the reports and recommendations of the Mental Health 
Implementation Task Forces. 





Recommendation 


To help ensure the efficient, effective, and appropriate use of Assertive 
Community Treatment (ACT) teams, the Ministry should: 


e determine the required number and distribution of ACT teams for the 
province; 


e monitor ACT teams to ensure that they are serving the seriously and 
persistently mentally ill target population; and 


e ensure there are adequate services available to meet the needs of 
individuals no longer requiring ACT services. 
Ministry Response 


As part of their mandated role, Mental Health Implementation Task Forces will 
be recommending future funding priorities for community services, including 





ACT teams. Information contained in the community assessment projects will 
inform the Ministry on the level of need and the services required. With the 
assessments completed at all current and former provincial psychiatric 
hospitals, the Ministry will have a very good basis for determining the number 
and distribution of future ACT teams and other required services. 


The Ministry has initiated the second year of a monitoring and outcome survey 
for all ACT teams and plans to continue this through a future, larger 
community mental health information system. 





Currently, the level of need for those no longer requiring the level of care 
provided by an ACT team must be assessed by the individual team. Case-by- 
case planning will be completed in conjunction with the current existing 
services in that area. 











FUNDING 


The need for community mental health services in a geographical area is largely dependent 
on the prevalence of serious mental illness in the local population. Many factors contribute 
to the incidence of mental illness. For example, research indicates that certain mental 
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illnesses are more prevalent in certain age groups, and poorer areas appear to have 
significantly higher rates of serious mental health problems. 


Ministry funding for community mental health programs is primarily historically based— 
that is, the amount of funding disbursed in any year is based on the amount disbursed in 
the previous year. Since 1992, there have been no increases in base funding provided to 
community mental health agencies for programs that were operating at that time. In that 
regard, in June 2001, one district health council, as part of its review of annual operating 
plans, noted that “as a result of not having had an increase to their base budget in several 
years, yet having to contend with such pressures as pay equity legislation, rising rental and 
utility costs, technology infrastructure requirements, increasing transportation costs and the 
need to offer competitive staff remuneration, community mental health agencies have been 
forced to reduce services to the seriously mentally ill in order to stay within existing base 


budgets.” 


A recent analysis performed by the Ministry identified significant variations in the per 
person funding for community mental health services among the different regions of the 
province. While the average provincial per capita rate in the 2001/02 fiscal year was $26, 
the per capita funding in six of the seven regions of the province ranged from approximately 


$11 to $34. The per capita funding in the seventh region was approximately $60. 
At the time of our audit, the Ministry had not developed a funding formula that takes into 


account the relative need for services and the costs of delivering services in the different 
regions of the province. Funding based on assessed need helps ensure that individuals with 
similar needs have access to similar services regardless of where they live in the province. At 
least two other jurisdictions outside Canada fund mental health services based on an 
assessment of need. 


Similarly, in its 1999 advice to the Minister, the Health Services Restructuring Commission 
stated that it “continues to support the importance of linking allocation of resources to 
benchmarks based in the characteristics of the population served (i.e., needs-based funding) 
in the medium to long term.” 





Recommendation 


To ensure that community mental health funding provided to regions and 
agencies is reasonable and equitable, the Ministry should develop a process 
that provides funding based on an assessment of services needed and of the 
resources required to meet those needs. 


Ministry Response 


The Ministry is in the process of converting the existing Community Mental 
Health (CMH) Budget & Inventory System to a CMH MIS (Wlanagement 
Information Systems) chart of accounts in the Ontario Hospital Reporting 
System. The MIS chart of accounts will provide the Ministry with more details 
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for in-year reporting using standard financial and statistical accounts and 
consistent definitions of mental health services. This will allow the comparison 
of expenditures across and within the various mental health sectors. 


The Ministry is also in the process of setting up the Common Data Set—Mental 
Health (CDS—MH). It employs the same set of mental health services that are 
defined in the CMH MIS chart of accounts. The collection of data for this data 
set will be enhanced substantially by aggregating and reporting client-level 
data using the client linkage system. 





The CDS—MH will assist the Ministry to move towards reasonable and 
equitable funding for community mental health agencies. 
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MINISTRY OF HEALTH AND LONG-TERM CARE 


3.04—Long-Term Care 
Facilities Activity 





BACKGROUND 


Long-term-care facilities provide care and services to individuals who are unable to live 
independently at home and who require the availability of round-the-clock nursing services 
to meet their daily nursing and personal care needs. These facilities comprise nursing homes 
and homes for the aged. They operate under the authority of the Nursing Homes Act, the 
Homes for the Aged and Rest Homes Act, and the Charitable Institutions Act. These acts and 
their regulations specify the requirements for admission, the care to be provided to residents, 
the rights of residents, the responsibilities of the facility, and the obligations of the Ministry. 
Rest and retirement homes, which may provide care, including assistance with daily living 
activities as well as nursing care, do not receive ministry funding and are not covered by 
these three acts. 


In each act, the stated fundamental principle to be applied is that each long-term-care 
facility “is primarily the home of its residents and as such it is to be operated in such a way 
that the physical, psychological, social, cultural, and spiritual needs of each of its residents 
are adequately met and that its residents are given the opportunity to contribute, in 
accordance with their ability, to the physical, psychological, social, cultural, and spiritual 


needs of others.” 


Admissions to long-term-care facilities are arranged by placement co-ordinators designated 
by the Minister. In determining eligibility for admission, a placement co-ordinator takes into 
account an assessment made by a health practitioner relating to a person’s impairment or 
capacity and an assessment of information relating to the person’s requirements for medical 
treatment, health care, or other personal care. 


The Ministry's key responsibility regarding the operations of long-term-care facilities is to 
ensure that they are delivering services to residents in accordance with their service 
agreements with the Ministry and in compliance with applicable legislation and ministry 
policies. Ministry oversight for long-term-care facilities is carried out through the Ministry’s 
seven regional offices. Funding for the Long-Term Care Facilities Activity is provided 
through the Ministry’s Integrated Health Care Program. | 
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In 1998, the government announced an eight-year plan to provide 20,000 new long-term- 
care beds and to renovate structurally non-compliant facilities containing 13,583 long- 
term-care beds. In March 1999, the government announced that the 20,000 new beds 
would be completed by 2004. With respect to structurally non-compliant facilities to be 
renovated by 2006, the number of beds was later revised to 15,835. In early 2000, the 
Ministry established a Long-Term Care Redevelopment Project office to undertake 
operational responsibility for the plan. 


Funding for long-term-care facilities is provided through four distinct per diems (funding 
envelopes). The daily rate for each is set by regulation. As at March 31, 2002, the per diem 
rates for residents of long-term-care facilities who required an average level of care were as 
outlined in the following table. 


Per Diems for Long-Term-Care Residents 
Requiring an Average Level of Care, 
as at March 31, 2002 




















* Facility costs, including administration, housekeeping, building 
and operational maintenance, and dietary and laundry services. 


Source of data: Ministry of Health and Long-Term Care 


The per diems are the same for all facilities, except for the nursing and personal care rate, 
which is based on an assessed level of care for each resident living in a facility. 


Residents make co-payments for their accommodation and food. The maximum daily rate 
paid by the resident for basic accommodation is $44.51. Residents in basic accommodation 
who do not have sufficient income to pay the maximum rate can apply for a rate reduction. 
The amount of the reduction is dependent on the resident’s income. Facilities are allowed to 
reduce the basic accommodation rate to $30.24 per day without obtaining ministry 
authorization. Any further reductions require ministry approval. 


If a resident pays less than the basic rate, the Ministry makes up the difference, so that the 
facility receives the full $44.51. A long-term-care facility can also charge residents an 
additional $8 for a semi-private room and $18 for a private room. Residents in semi-private 
and private rooms must pay the full basic accommodation rate in addition to the applicable 
premium. 
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As of March 2002, there were approximately 60,000 residents in 558 long-term-care 
facilities. For the 2001/02 fiscal year, long-term-care facilities received approximately 
$1.6 billion in funding from the Ministry and approximately $793 million in 


accommodation charges from residents. 


AUDIT OBJECTIVES AND SCOPE 


The objectives of our audit of the Long-Term Care Facilities Activity were to assess whether 
the Ministry had adequate procedures in place to: 


* ensure that resources were managed with due regard for economy and efficiency; 


* ensure that facilities providing long-term-care were complying with applicable 
legislation and ministry policies; and 


¢ measure and report on the Activity’s effectiveness. 


Our audit was performed in accordance with standards for assurance engagements, 
encompassing value for money and compliance, established by the Canadian Institute of 
Chartered Accountants, and accordingly included such tests and other procedures as we 
considered necessary in the circumstances. Prior to the commencement of our work, we 
identified the audit criteria that would be used to address our audit objectives. These criteria 
were reviewed and agreed to by senior management. 


The scope of our audit work, which was substantially completed by May 2002, included a 
review and analysis of information available at the Ministry’s head office, three regional 
offices, and the Redevelopment Project office, as well as discussions with appropriate staff. 
We visited a number of long-term-care facilities to gain a better understanding of the 
services being provided and to review certain procedures. 


We also met with representatives of the Ontario Long Term Care Association and the 
Ontario Association of Non-Profit Homes and Services for Seniors, which collectively 
represent most of the owners/operators of long-term-care facilities in Ontario. 


In 2001, the Ministry’s Internal Audit Services Branch undertook a review of a long-term- 
care facility regarding financial complaints and compliance with the Nursing Homes Act. 
However, the results of this review did not affect the extent of our audit work because it 
primarily addressed specific concerns related to one facility. 


OVERALL AUDIT CONCLUSIONS 


In certain significant respects, the Ministry did not have all of the necessary procedures in 
place to ensure that long-term-care resources are managed with due regard for economy 
and efficiency and that long-term-care facilities are complying witirapplicable ministry 
policies. A number of our concerns were also reported in our 1995 Annual Report. In 
particular, we noted: 
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¢ The Ministry had still not developed either standards to measure the efficiency of 
facilities in providing quality care or models for staff mixes for providing nursing and 
personal care and, therefore, did not have a sufficient basis for determining appropriate 
levels of funding. In addition, the Ministry had not addressed the results of a 2001 
consulting report that noted that residents of Ontario’s long-term-care facilities “receive 
less nursing and therapy services than [those in] similar jurisdictions with similar 
populations.” 


¢ Although the Ministry inspected all long-term care facilities in 2001, it did not have a 
risk-based approach for prioritizing its facility inspection procedures, such as conducting 
in-depth inspections of facilities with a history of failing to meet provincial quality-of- 


care standards. 


¢ Some regions lacked advisors, such as dieticians, who could provide specialized advice. 
For instance, in facilities that they inspected, dieticians found higher incidences of 
unmet dietary criteria—such as unsanitary procedures in the kitchen and lack of 
appropriate nutrition—than compliance advisors found. 
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¢ The Ministry was not adequately tracking complaints, unusual occurrences, and 
outbreaks of contagious diseases to identify and resolve systemic problems. In 2001, of 
seven regions, only two regions recorded unusual occurrences, which totalled 1,900. In 
the same year, only four regions recorded outbreaks of contagious diseases, which 


totalled 219 and affected 7,500 residents and staff. 


* Contrary to legislation, none of the nursing homes in Ontario had current ministry- 
issued licences at the time of our audit. At least 15% of licences had expired more than 
one-and-a-half years ago. As well, most nursing homes that opened after 1998 had 
never been issued a licence. 


¢ Surplus funds were not being recovered from facilities on a timely basis because the 
Ministry was performing annual reconciliations almost two years after the applicable 
year-end. The delayed recovery of approximately $50 million for the 1999 calendar 
year resulted in approximately $5 million in interest expenses being passed on to the 
taxpayers. 


Since our previous audit in 1995, the Ministry has established a target for the number of 
long-term-care beds required and has implemented a strategy to cope with the increasing 
demand for beds arising from the growing elderly population. However, the Ministry did 
not have a process in place for periodically reviewing whether its target of 100 beds per 
1,000 individuals aged 75 and over was appropriate. 


Through the long-term-care redevelopment project, the Ministry allocated funding to 
build new long-term-care facilities in regions of the province where the need for additional 
beds was the greatest and was providing financial assistance to facilities that do not meet 
minimum structural and environmental standards. 
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Finally, we concluded that the Ministry's procedures for providing adequate accountability 
to the public and ensuring that long-term-care facilities provide services efficiently and 
effectively were impaired because: 


¢ Financial information submitted by facilities was not sufficient to allow the Ministry to 
determine whether funds had been used in accordance with the Ministry’s expectations. 


¢ The Ministry had not developed outcome measures that addressed the appropriateness 
of services provided, including the quality of care received by residents. 


DETAILED AUDIT OBSERVATIONS 
MONITORING QUALITY OF CARE 


The Ministry's primary tool for monitoring the quality of care provided to residents of long- 
term-care facilities is its Compliance Management Program, whereby the Ministry is to: 


* conduct annual inspections of all long-term-care facilities; 


* conduct other inspections as required by specialists, such as dietary and environmental 
advisors; and 


* investigate complaints submitted by residents, residents’ families, and the general public. 


The Compliance Monitoring Program is the responsibility of the regional offices. Each 
regional office is staffed by a long-term-care manager and compliance advisors who are 
registered nurses. Some regional offices also have an environmental and/or dietary advisor(s) 
to handle more specialized reviews. Compliance advisors inspect facilities to ensure that all 
legislation, regulations, and standards as defined in the Ministry's Long-Term Care Facility 
Program Manual are being met. Facilities that fail to meet any of these requirements must 
take appropriate and timely corrective action. If standards continue to be unmet and/or 
there is a serious threat to the health, safety, or welfare of residents, a facility can be put 
under enforcement. 


In July 2000, the Minister announced that a full evaluation of the Compliance 
Management Program would be conducted to: 


* identify the current strengths and weaknesses of the program; 

* analyze long-term-care monitoring mechanisms in other jurisdictions; 

* identify opportunities to strengthen linkages with other long-term-care initiatives; 
* recommend opportunities for program improvement; and 


* recommend areas for more in-depth study. 
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The review was being done internally in consultation with stakeholder groups. Initially the 
deadline for the project was December 2001, but has been rescheduled for the end of 
2002. 


At the time of our audit, the review committee had identified the strengths and weaknesses 
of the compliance program and developed a draft enforcement policy and a list of triggers 
to identify when a facility may not be spending funds prudently or for the purposes 
intended. However, recommendations for compliance program improvement remained to 


be developed. 


Annual Inspections 
Under the Nursing Homes, Homes for the Aged, and Charitable Homes acts, the Ministry has 


the right to inspect facilities to ensure compliance with legislation and regulations, service 
agreements, and/or licences. According to ministry policy, “reviews will be conducted at 
least once in a calendar year.” The objective of the annual inspection is to monitor and 
evaluate the quality of resident care and services, the quality of programs, and the overall 
operation of each facility. The results of the annual inspections are to be posted in each 
facility. 

Between 1997 and 1999, fewer than half of the facilities were inspected annually. In 2000, 
the then Minister announced that all long-term-care facilities would be inspected on a 
yearly basis. All of the facilities were inspected in the 2001 calendar year. We reviewed the 
inspection process and noted that, although senior management tracks the number of 
inspections completed, they do not routinely review the findings of these inspections in 
order to identify: 


¢ — facilities not in compliance with legislation and ministry policies; 


¢ — facilities with infractions that can have a serious negative impact on the quality of 
resident care; 


¢ — facilities that consistently have problems; and 
* operators who demonstrate a pattern of non-compliance in their facilities. 
We also noted that: 


¢ The Ministry lacked a formalized risk-assessment approach for prioritizing inspection 
procedures or focusing on facilities with a history of failing to meet provincial quality 
standards. Currently, the Ministry conducts an in-depth, annual inspection at all 
facilities, which on average takes four days for each facility. However, a full inspection 
may not be warranted for facilities that have historically always been in compliance and 
have received few or no complaints, while facilities with chronic problems may warrant 
more in-depth inspections. 


¢ Both associations representing facility operators informed us that, according to their 
member facilities, compliance standards were not being applied consistently among 
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facilities and across regions. Much depends on how those standards are interpreted and 
enforced by compliance advisors. For example, some compliance advisors told us that 
they might give an operator a warning while another advisor might give a citation for 
the same unmet standard. 


* A facility may be notified up to a week in advance of an upcoming annual inspection. 
Some facilities may use this time to “prepare” for the inspection. As such, the inspection 
results may not be reflective of the ongoing care provided at the facility. For example, 
one facility wrote to its corporate office that, “The Nursing Department, with 
considerable help from Corporate staff and registered staff from other Corporate 
facilities, had a week to prepare for the [Ministry] annual review. All staff involved are to 
be commended for their efforts which were reflected in the favourable outcome of the 
review.” 


* Ministry policy states that a compliance advisor can refer issues to advisors who specialize 
in the areas of environmental health, dietary, financial, and medical services, We were 
advised that, while there has not been a medical advisor available for the last five years, 
medical advice is sought when it is needed. However, none of the three regions we 
visited had an environmental advisor and only one had a dietary advisor. In addition, the 
compliance advisors lacked guidance as to when a referral was appropriate. We also 
noted that regions with dietary advisors were more likely to initiate dietary inspections, 
and facilities inspected by dieticians had higher reports of incidences of unmet dietary 
criteria. [hese included unsanitary procedures in the kitchen, lack of disinfectant use, 
inadequate maintenance of food temperatures, and lack of appropriate nutrition. 


* “Focused audits” are part of the annual inspection process and require compliance 
advisors to select the case files of six residents from each facility to investigate areas 
considered to be of highest risk to residents, including unplanned weight change, 
pressure ulcers, pain and discomfort, disruptive behaviour, and use of restraints, The 
results of these audits are used to reach conclusions on the adequacy of care provided by 


a facility. 


The sampling methodology was left to the discretion of each compliance advisor. Some 
compliance advisors told us that they based sample sizes on previous facility visits and 
personal comfort levels, while others indicated they selected samples based on bed 
capacity. We reviewed a sample of annual review files and noted that in 92% of the cases 
less than six case files were reviewed. Furthermore, we found no evidence to indicate 
that management had reviewed and approved the adequacy of the sample sizes used. A 
similar concern was raised in our 1995 Annual Report. 










Recommendation 


To help ensure that long-term-care facilities meet the assessed needs of each 
of their residents, the Ministry should: 
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* ensure senior management assesses the results of annual facility 
inspections for possible corrective and preventive action; 


¢ implement a formalized risk-assessment approach for its annual 
inspections that concentrates on facilities with a history of non-compliance 
and prioritizes inspection procedures; 


e ensure consistency in the application of standards; 


¢ establish acceptable notification periods and conduct surprise inspections 
of high-risk facilities to reduce the risk that facilities will “prepare” for an 
inspection; and 


¢ evaluate the experience and skills required to inspect facility operations 
and ensure the appropriate mix of specialists is available. 


Ministry Response 


The Ministry's compliance program is carried out by staff who are fully 
committed to ensuring the health and quality of life of residents. However, the 
Ministry strives to improve and is currently undertaking a comprehensive 
review to determine areas for improvement, including appropriate staff mix. 
The Ministry continues to recruit for vacant positions to meet the needs of its 
compliance management program. 
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The core of this review is the development of a risk-assessment approach. 
Features may include the addition of risk indicators that will act as triggers for 
action by compliance staff and/or senior management. These indicators and 
actions would be in addition to actions that are currently triggered by reports 
from the compliance management team. 





The Ministry currently conducts unannounced visits to long-term-care facilities 
and expects to incorporate surprise inspections in the future. 














Health and Safety of Residents 


COMPLAINTS 

Ministry policy requires that all complaints made against a long-term-care facility be 
investigated within 20 days of being received. Currently all complaint investigations are 
conducted by regional compliance officers. In the past, the Ministry had an enforcement 
unit, which investigated the more serious complaints, such as alleged resident abuse. 
However, the enforcement unit was discontinued in June 1999. 


The Ministry tracks the response times for and the nature and source of the complaint. We 
analyzed complaint information recorded in the Facility Monitoring Information System 


(FMIS) and noted the following items: 


¢ In the last two years, 83% of complaints were investigated within the required 20 days. 
However, performance varied greatly across regions. While one region almost always 
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met the 20-day requirement, another region was consistently late in completing 
one-third of its complaint investigations. 


* In the three regions we visited, at least half of the facilities did not receive any complaints 
in 2001. We noted that most facilities that had received more than 10 complaints each 
also had significant non-compliance issues identified during the annual inspection 
process. For example, a facility that received a total of 29 complaints in 2001 and 24 
complaints in 2000 also had reoccurring non-compliance issues going back to 1990. 
However, the Ministry had no procedures for analyzing complaints in order to identify 
and address facility/system weaknesses. 


UNUSUAL OCCURRENCES 


The Ministry defines an unusual occurrence as “an occurrence that poses a potential or 
actual risk to the safety, security, welfare, and/or health of a resident, or to the safety and 
security of the facility, which requires action by staff.” Facilities are required to report all 
unusual occurrences to the Ministry's regional offices. The regional compliance advisors 
review the details of incidents, and consider what action, if any, is necessary. 


At the time of our audit, there was no requirement to record unusual occurrences in FMIS. 
However, for the two regions that did record unusual occurrences in EMIS, we found a 
total of 1,600 unusual occurrences for calendar year 2000 and 1,900 for 2001. Injuries 
requiring transfer of a resident to a hospital for treatment or admission accounted for 70% 
of all unusual occurrences recorded in FMIS while cases of alleged abuse of residents 
represented 9% of reported cases. 


We reviewed unusual occurrence reports for a sample of long-term-care facilities in the 
regions we visited and found that facilities that exhibited a higher number of repetitive, 
serious non-compliance issues and complaints also experienced a higher number of unusual 
occurrences. For instance, one such facility that had ongoing serious compliance issues for 
more than 10 years also had very serious occurrences. 


OUTBREAKS OF CONTAGIOUS INFECTIONS 


Ministry policy requires long-term-care facilities to report all outbreaks of contagious 
infections to their respective regional offices. This is to help the Ministry: 


* identify outbreak prevalence and emerging trends in long-term-care facilities; 
* ensure outbreak information is relayed to appropriate personnel; and 


* ensure that information relating to outbreaks is complete, including information 
outlining the duration of the outbreak. 


The Ministry relies on local public health departments to assess and respond to outbreak 


information. 


For each reported outbreak, an outbreak document is prepared, reviewed by the regional 
compliance advisor, and filed in the appropriate facility’s file. Regional offices are required to 
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maintain an outbreak log, which lists each facility that had an outbreak, the type of 
outbreak, and the dates of initial and final notification to the Ministry. 


At the time of our audit, only four of the seven regional offices were recording outbreaks in 
PMIS. Those four regions alone reported a total of 219 outbreaks in 2001, affecting over 
7,500 residents and staff. We visited two of the regional offices that did not maintain their 
logbooks on FMIS. The logbook for one region indicated an additional 80 outbreaks in 
2001 but contained no indication of how many people were affected. The second region 
we visited did not maintain a logbook at all. 


For the regions that did record outbreaks in FMIS, we noted one region accounted for 
62% of all outbreaks in 2001. This trend was consistent with the prior three years. 
However, it was unclear whether there was a problem with infection control practices in this 
region or whether facilities in other regions were not accurately reporting all outbreaks to 
their regional offices. In addition, we found that no effort had been made to correlate the 
number of outbreaks per facility with the unmet criteria identified in annual inspections, 


such as poor sanitation, environmental cleanliness, or resident hygiene. 





















Recommendation 


To better protect the health and safety of residents of long-term-care facilities, 
the Ministry should ensure that all: 


¢ complaints are investigated and responded to in a timely manner; 


e unusual occurrences and outbreaks of contagious infections are reported 
to the Ministry and recorded in its Facility Monitoring Information System 
on a timely basis; and 

¢ complaints, unusual occurrences, and outbreaks of contagious infections 
are assessed in relationship to annual facility inspection results to identify 
and resolve systemic problems. 





Ministry Response 


The Ministry has a policy of investigating and responding to complaints within 
20 days. The Ministry has a good track record of respending to complaints; 
however, it is continually striving to improve. 


Ministry staff follow up on all unusual occurrences. The Ministry, in conjunction 
with local public health agencies, has strict protocols and procedures to 
ensure resident safety in outbreak situations (for example, quarantines and 
specific hygiene measures). Local public health agencies determine when a 
facility is in outbreak status. 


The Ministry is committed to improving its data-entry systems with respect to 
these issues and will use this data to identify and resolve any systemic 
problems. 
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Facility Licences and Service Agreements 


Under The Nursing Homes Act “no person shall establish, operate, or maintain a nursing 
home except under the authority of a licence issued by the Director under this Act.” All 
licences expire and must be renewed annually and may be revoked or their renewal refused 
if the licensee contravenes the legislation. While there are no licensing requirements for 
municipal and charitable homes, all long-term-care facility operators are required to have a 
signed service agreement with the Ministry. 


We reviewed the licence status of nursing homes and found that, at the time of our audit, 
none of the nursing homes operating in the province of Ontario had a valid operating 
licence. While most of the licences had expired within the last year, at least 15% of licences 
had expired more than one and a half years ago. In fact, we noted one facility whose licence 
had expired in 1994, another whose licence had expired in 1997, and two others whose 
licences had expired in 1998. As well, most nursing homes that opened after 1998 had 


never been issued a licence to operate. 


According to the Management Board of Cabinet Directive on Transfer Payment 
Accountability, a signed agreement must be in place prior to advancing any provincial funds 
to transfer-payment recipients, which include long-term-care facility operators. We noted, 
however, that the Ministry’s normal practice is to finalize and sign service agreements after 
the funding year has passed. At the time of our audit, none of the service agreements for the 
2001 and 2002 calendar years had been signed. While ministry staff indicated to us that 
the old service agreements remain in effect, the facility operators in one of the regions we 
visited often disagreed with the Ministry’s position. Service agreements cover a specific year 
because they indicate the amount of the subsidy only for that year. 


We also noted the Ministry did not take into consideration whether or not the facilities were 
in compliance with ministry standards at the time service agreements were signed. 





a 
Recommendation 


To help ensure that ministry policies and legislation regarding long-term-care 
facilities are followed and that long-term-care service providers understand 
their responsibilities, the Ministry should ensure that all long-term-care 
facilities have valid service agreements and that each facility’s compliance 
status is taken into account. 


The Ministry should also ensure that all nursing homes have valid licences as 
required by legislation. 
Ministry Response 


The Ministry is undertaking a review of the management process supporting 
service agreements. The 2001 and 2002 service agreements will be distributed | 
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to facility operators by October 31, 2002. The Ministry will distribute the 2003 
service agreements to facility operators by December 31, 2002. 


The Ministry now has a program in place to ensure all licences are current. As 
of September 1, 2002, all nursing homes have current licences. 








PER DIEM FUNDING 


Level-of-care Classifications 


Level-of-care requirements for residents are determined through an annual classification 
assessment of residents at each facility conducted by ministry assessors. The assessors, who are 
registered nurses with long-term-care experience, conduct the assessments by reviewing 
resident charts and plans of care. Residents are classified into one of seven nursing and 
personal care categories based on the level of care they require. Each category is assigned an 
established weight based on assessed resident needs, taking into account the related resource 
use. The percentage of residents in each category is multiplied by weighting factors to 
determine the case-mix measure of that facility. The ratio of the facility’s case-mix measure to 
the average provincial case-mix measure produces a case-mix index, which is multiplied by a 
set per diem rate to determine the funding a facility will receive per resident for nursing and 
personal care. 


Long-term-care facility funding through the nursing and personal care funding envelope in 
the 2001/02 fiscal year totalled approximately $1.1 billion of the $1.6 billion provided by 


the Ministry to long-term-care facilities. 


LEVEL-OF-CARE AUDITS AND APPEALS 


A decrease in a facility’s case-mix measure affects the per diem it receives for nursing and 
personal care. A facility is entitled to appeal its classification if its case-mix measure decreases 
by more than 7% from the previous year. In 1997, the Ministry introduced annual audits 
of facilities whereby it verifies the level-of-care classification at a sample of facilities. These 
audits initially involve the reassessment of a minimum of 20 residents and are based on the 
residents’ documented care and direct observation. If the case-mix measure for those 
residents exceeds their annual classification by more than 10%, a full audit is conducted on 
all facility residents who were there when the annual classification was completed. 


According to ministry policy, if the full audit verifies that the documented and observed 
care have changed by more than 10% from the annual classification, the Ministry will 
consider increasing or decreasing funding. In a December 2001 memo to all regional 
offices, the Ministry noted that since the audit and appeals process began, the policy to 
decrease funding had not been applied where warranted. 
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The Ministry currently offers seminars to help facilities improve their documentation of 
resident-care needs but does not penalize those facilities that, based on the audits, it knows 
or strongly suspects are misrepresenting resident needs in an effort to obtain additional 


funding. 





f 


Recommendation | 


To help ensure fairness in the levels of funding provided to long-term-care 
facilities, the Ministry should adjust funding where warranted as a result of any 
level-of-care classification audit in accordance with its policy. 


Ministry Response 


Currently, some adjustments are made by the Ministry as a result of audits. 
However, the Ministry will be reviewing the implementation of the policy to 
ae funding as a result of the audit and appeals process. 














Reasonableness of Per Diem Funding 


In our 1995 audit report, we recommended that the Ministry use information on the cost 
of providing care and accommodation to verify the accuracy of the standard rates paid in 
each of the funding envelopes. Since that time, the Ministry has made periodic funding 
adjustments based primarily on the overall increase in the provincial case-mix measure. 
However, the Ministry has not done a detailed analysis to determine an appropriate amount 


of funding. 


In June 2000, the Ministry established a committee to review how funding is determined, 
allocated, and distributed and to recommend improvements to the current per-diem-based 
methodology. The committee, composed of ministry staff and representatives from 
associations of service-provider groups, recommended increases in the four funding 
envelopes based on changes in residents’ needs, negotiated contract increases with staff, and 
other inflation-related adjustments but did not discuss the adequacy of the current funding 
amount. 


For the past two years, the Ministry has produced sector per diem reports that list for each 
facility the actual amount spent per resident per day for each type of expense and for each 
funding envelope in total. The cost data is accumulated by sector (charitable homes, 
municipal homes, and nursing homes) within each region. We reviewed the 1999 reports 
for the three regions we visited and noted large variances among the sectors and among 
facilities in expenditures for nursing and personal care and in accommodation expenses. 


In our 1995 Annual Report, we also recommended that the Ministry develop standards to 
measure the efficiency of facilities in providing quality care and develop models for staff 
mixes for providing nursing and personal care to arrive at appropriate funding levels. Prior 
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to 1996, the Ministry required each long-term-care facility to have a registered nurse on 
duty and on site at all times, and it guaranteed, as a minimum, sufficient funding to ensure 
that each nursing-home resident received, on average, a minimum of 2.25 hours of nursing 
and personal care per day. This funding was to be provided regardless of the overall care 
needs of residents in each nursing home. 


In 2001, the Ministry provided the funding for a consulting firm to review how the level of 
services provided to residents of Ontario’s long-term-care facilities in terms of nursing staff, 
health care aides, and therapies compared to services provided in other jurisdictions in 
Canada, the U.S., Europe, and in Ontario’s chronic-care facilities. The report considered 
only the amount of care provided, not the quality of care. According to the consultants, the 
study’s limitations included the facts that data for many of the comparative jurisdictions were 
gathered from three to five years earlier than the Ontario data and that “several of the 
jurisdictions were required to submit the data for funding purposes, which may influence 


data quality.” Nevertheless, the published report stated: 


The results of this study indicate that residents in Ontario [long-term-care] facilities receive 
less nursing and therapy services than similar jurisdictions with similar populations. 
Furthermore, Ontario [long-term-care] residents have some significant differences in terms 
of their levels of depression, cognitive levels and behavioural problems which indicate 
higher needs for service levels to meet higher care requirements. 


Specifically: 


¢ The proportion of care provided by registered nurses in Ontario’s long-term-care 
facilities to each resident per day was the lowest in comparison with other jurisdictions. 


¢ Only one-third of Ontario residents in long-term-care facilities who had restricted 
ranges of motion received any range-of-motion exercise. 


¢ Ontario residents in long-term-care facilities had the highest proportion of mental 
health disturbances and/or problems, of which 65% were handled either with restraints 
or anti-psychotic medication. Less than 6% had any intervention related to evaluation or 


“talk therapies.” 
We found no evidence to indicate that the Ministry had addressed the results of this study. 
We also noted that 36 U.S. states have established staffing requirements or standards, 


such as: 

¢ aminimum number of hours of nursing care per resident per day; 
¢ a minimum caregiver-to-resident ratio; and/or 

¢ arequirement for a registered nurse to be on site 24 hours a day. 


Currently, the Ministry does not have any staffing requirements and does not track facility 
staff-to-resident ratios, the number of registered-nursing hours per resident, or the mix of 
registered and non-registered nursing staff. 
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Recommendation 





To help ensure that the funding provided to long-term-care facilities is 
sufficient to provide the level of care required by residents and that the 
assessed needs of residents are being met, the Ministry should: 


e verify the reasonableness of the current standard rates for each funding 
category and develop standards to measure the efficiency of facilities 
providing services; 

* track staff-to-resident ratios, the number of registered-nursing hours per 
resident, and the mix of registered to non-registered nursing staff and 
determine whether the levels of care provided are meeting the assessed 
needs of residents; and 


e develop appropriate staffing standards for long-term-care facilities. 


Ministry Response 


Effective August 1, 2002, the Ministry increased nursing and personal care 
funding by $100 million ($6.33 per resident per day). The total per diem for a 
long-term-care facility with average care levels is $110.73. 


Currently, long-term-care facilities receive differential funding based on the 
care needs of their residents. The Ministry is committed to investigating the 
feasibility of implementing a new resident-classification instrument and 
funding methodology that will further enhance its ability to assess resident 
care and staffing needs. 











Annual Reconciliations 
The Ministry's Long-Term Care Facility Program Manual states that each facility is required 


to submit an audited annual report to its regional office so the regional office can compare 
actual to approved expenditures and determine whether a final funding adjustment is 
necessary. 


We noted that the Ministry had not established any timelines for completing the annual 
reconciliations and that surpluses were not being recovered promptly. For instance, 
surpluses for 1998 were recovered in late 2000 and early 2001, and, at the time of our 
audit, reconciliations for 1999 were still being completed. In each of the last three years, net 
surpluses owed to the Ministry exceeded $50 million. Given that the average one-year 
borrowing rate for the government in 2000 and 2001 was 5.7% and 3.8% respectively, the 
late recoveries of excess funding for the 1999 calendar year alone have cost taxpayers at least 
$5 million in interest expenses. 


For the 1999 calendar year, the Ministry did not forward annual report forms to facilities 
until April 2001. Most facility operators completed and returned the required forms within 
90 days of receipt, as required under ministry policy. However, regional office staff did not 
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begin the reconciliation process until the fall of 2001. The annual report forms for the 
2000 calendar year were sent to operators on February 25, 2002, 14 months after that 
year-end. 


Most long-term-care facilities have surpluses because the Ministry regularly underestimates 
facility revenue from resident payments. For example, the Ministry used actual resident 
revenue for 1996 to estimate facility resident revenue for 1999, without making 
adjustments for known increases in the per diem accommodation rates. Resident revenue 
cannot be easily estimated because some residents are eligible for rate reductions due to 
financial need. As mentioned earlier, facilities are allowed to reduce the basic resident rate 
for ward accommodation from $44.51/day to $30.24/day for residents in financial need 
without ministry authorization. 


Although financial analysts in the regional offices are now adjusting in-year payments based 
on more recent estimates of resident revenue, there were inconsistencies in the approaches 
among regions. Ministry staff advised us that, beginning July 1, 2002, the process for in- 
year adjustments will be standardized. 


RECONCILIATION PROCESS 


Punding provided to long-term-care facilities within each envelope is intended to cover 
eligible expenses as outlined in the Ministry’s Long-Term Care Facility Program Manual. 
Except for funds received for certain accommodation costs, facilities must return all unspent 


funds to the Ministry. 


We reviewed the financial information submitted by facilities to the Ministry’s regional 
offices and found that there was insufficient information to determine whether funds within 
each envelope were used for their intended purposes. Most of the Ministry’s regional 
financial analysts we surveyed agreed that the information was inadequate. 


The majority of expenditures in the nursing and personal care funding envelope are for staff 
salaries, yet facilities are not required to submit staffing data, such as the number of 
employees per type (registered nurses, practical nurses, and health care aids) or their 
respective salaries. Also, facilities do not routinely provide regional offices with a list of 
equipment and supplies purchased during the year under each funding envelope. Analysts 
advised us that, in several instances, facilities charged accommodation equipment and 
supplies as medical items to the nursing and personal care funding envelope, thus 
minimizing the amount of funding they may be required to return to the Ministry. 


In addition, a number of long-term-care facility operators are also operating retirement 
homes on adjacent premises. Retirement homes do not receive ministry funding but may 
share some services and staff with their affiliated long-term-care facilities. For three regions 
alone, the Ministry identified 69 long-term-care facilities with attached retirement homes 
operating in the same facility. Long-term-care facilities are not required to submit a full set of 
audited financial statements. Without audited financial statements that contain segregated 
information, the Ministry does not know whether operators are properly allocating costs. 
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Recommendation 











To help ensure surplus funding to long-term-care facilities is accurately 
identified and returned to the province on a timely basis, the Ministry should 
ensure that: 


¢ audited financial information provided by facilities meets ministry needs; 
and 
° reconciliations are completed and surpluses recovered on a timely basis. 


Ministry Response 


The Ministry will review the form and content of information currently collected 
from facilities to ensure that it is meeting ministry needs. 


The Ministry will commit to sending out the 2001 and 2002 reconciliation 
reports to long-term-care facilities by December 31, 2002. 


The Ministry has developed a consistent revenue/occupancy report. Beginning 
January 1, 2003, all seven regional offices will monitor and adjust cash flows as 
required. 








PS | 





THE LONG-TERM CARE REDEVELOPMENT 
PROJECT 


Supply of Long-term-care Beds 


In our 1995 Annual Report, we noted that, although it was aware of significant growth 
projected for the population aged 65 and over, the Ministry did not have a strategy for 
dealing with the anticipated increase in demand for long-term-care beds. We also noted that 
it did not have a systematic plan to determine where beds were most needed and to 
eliminate the wide variations in bed supply to make it equitable throughout the province. 


In the fall of 1996, the Ministry established the Long-Term Care Bed Distribution and 
Needs Study Steering Committee (steering committee), made up of ministry staff and 
representatives from external associations. The steering committee was to review ways to 
equitably distribute long-term-care beds throughout the province and to suggest any policy 
or legislative changes to facilitate such a distribution. 


In the introduction to its April 1997 report, the steering committee noted that provincially 
funded and regulated long-term-care beds had “for many years been inequitably distributed 
across Ontario both by region and within regions themselves. This distribution has occurred 
more by ‘accident’ than as a result of needs-based planning.” 


The steering committee indicated that “given the time parameters of the study and issues 
affecting the [long-term-care] facility service system, [it] was not able to establish a bed 
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planning target.” It recommended that the Ministry undertake further study and research 
to define a bed-planning target for long-term care. The steering committee also urged that 
its recommendations be considered in conjunction with recommendations regarding long- 
term care from the Health Services Restructuring Commission. In 1997, the Health 
Services Restructuring Commission released a discussion paper stating that by 2003, the 
province would need an additional 15,404 long-term-care beds. According to the 
Commission, these additional beds would result in an average bed ratio of 96.4 beds for 
every 1,000 individuals aged 75 and over. 


In April 1998, Management Board of Cabinet approved in principle a plan from the 
Ministry requesting a total of 20,000 new beds to be built by the end of the 2005/06 fiscal 
year and the renovation of facilities with 13,583 existing long-term-care beds that did not 
meet current structural requirements. Ministry staff informed us that although there was no 
ministry standard for determining the future need for long-term-care beds, they were 
attempting to reach a target of 100 beds for every 1,000 individuals aged 75 and over. 
While the Ministry was unable to provide information on how it arrived at this target, it was 
consistent with the target recommended by the Health Services Restructuring Commission. 
By way of comparison, a ministry-initiated study indicated that in 2000, Alberta had a ratio 
of 109 beds per 1,000 individuals aged 75 and over, while Manitoba’s planning guideline 
was 120 beds per 1,000 individuals aged 75 and over. It is important to note that the future 
need for long-term-care beds is affected by many factors, including the availability of home- 
care, chronic-care, and other services. 


Based on the allocation of the approximately 20,000 new beds, by 2006 the projected bed 
ratios across provincial service areas (municipalities, districts, or counties) are expected to 
range from 88 to 138 beds for every 1,000 individuals aged 75 and older. Without the 
20,000 new beds, projected bed ratios would have ranged from 38 to 138 beds for every 
1,000 individuals aged 75 and over. Areas of the province most likely to exceed the target 
were generally above the target before the new beds were allocated and, accordingly, were 
generally not awarded new beds. 


| Recommendation _ 


To help ensure that the need for long-term-care beds is met on a timely basis, 
the Ministry should: 








¢ conduct research to determine whether its target of 100 beds per 1,000 
individuals aged 75 and over is appropriate; and 


e develop a strategy to address the results of the research. 





Ministry Response 


The Ministry is currently conducting policy work on a long-term strategy for 
long-term care. This strategy will look at the full range of services available to 


seniors and make recommendations about programmatic responses. 





Sd 








Long-Term Care Facilities Activity 131 


VFM Section 3.04 





| VFM Section 3.04 | 


Capital Redevelopment Plan 


In March 1999, the government announced that the expansion project for long-term-care 
beds was being accelerated and that the 20,000 new beds would be completed by 2004, 
bringing the total number of long-term-care beds to approximately 77,000. The 2006 
deadline for renovating existing substandard facilities remained unchanged while the 
number of beds in substandard facilities was later revised from 13,583 to 15,835. 


Contracts for the construction and operation of new facilities for the 20,000 new beds were 
primarily issued through three competitive selection processes in December 1998, March 
1999, and May 2001. The 20,000 new beds included approximately 1,200 beds that had 
been awarded to operators in the 1980s and had not yet been developed. The regional 
offices handled the 1998 and 1999 selection processes based on common procedures 
established by the Ministry's head office. The third was undertaken by the newly established 
Long- Term Care Redevelopment Project office (Redevelopment Project office). 


The Redevelopment Project office was established by the Ministry in May 2000 to oversee 
the operation of its bed-expansion project, including: 


* developing and tracking the awarding of beds (including beds awarded in 1998 and 
1999); 


* reviewing and ranking the proposals in 2001; 
* negotiating development contracts with successful applicants; 
* reviewing and approving construction plans for new and replacement beds; and 


* advising on and resolving municipal issues related to the construction of the new beds. 


CAPITAL FUNDING 


The Ministry is funding the construction of facilities for the 20,000 new beds and the 
renovation of facilities containing 15,835 beds by providing a per diem of up to $10.35 per 
bed. These payments are for 20 years and commence the day the bed is available for use 
and will amount up to approximately $75,000 for each new or renovated bed. Over 20 
years, the capital costs to the Ministry will total up to $1.5 billion for the new beds and up 
to $1.2 billion for the renovated beds. 


In addition, in January 2001, the Ministry revised its policy on preferred accommodation 
revenue. Previously, facilities shared preferred-accommodation revenue (that is, additional 
amounts charged for private or semi-private accommodation) with the province on an equal 
basis. All operators, including those with existing facilities that were not being renovated, 
would now be permitted to keep 100% of their preferred accommodation revenue. 
According to ministry documentation, “This funding is intended to supplement the $10.35 
provided to facilities to support the cost of construction.” Currently, this will provide at least 
$50 million of additional annual funding to long-term-care facilities. 
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We noted that the Ministry had not assigned to either the Redevelopment Project office or 
the regional offices the responsibility for obtaining actual expenditure reports for facilities 
that had been operating for more than one year to ensure that per diem construction 
funding was based on actual costs. In February 2002, the Ministry assigned that 
responsibility to the regional offices. However, by that time, one facility was more than two 
years late in submitting its report. 


Recommendation 


The Ministry should ensure that the per diem paid to long-term-care facilities 
for capital construction are consistent with the actual construction costs 
incurred. 


Ministry Response 


The Ministry’s policy for funding construction costs of long-term-care facilities 
and development agreements require operators to submit audited statements 
of final capital costs. The Ministry has developed guidelines for consistent 
review and approval of these statements and will ensure they are appropriately 
implemented. * - a 





ALLOCATION OF NEW BEDS 


In its presentation to Management Board of Cabinet, the Ministry stated that “all 20,000 
new beds [were to be] awarded through a highest quality RFP process.” At the time of our 
audit, new beds had been allocated as outlined in the following table. 


Beds Awarded through 
Competitive Selection Processes, 
1998-2001 











* includes 1,065 beds returned by successful applicants and 
subsequently re-allocated to other applicants, and 704 beds 
allocated through subsequent competitive processes to meet 
needs in under-serviced areas 


Source of data: Ministry of Health and Long-Term Care 
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In the 1998 process, beds were to be awarded to the applicants who received the highest 
rankings after taking into account the Ministry's “policy decisions about appropriate 
geographical distribution of long-term-care facilities in each service area of the province of 
Ontario.” The Ministry could also require as a condition of award that a successful applicant 
develop beds in a different geographical location than described in the applicant’s proposal 
but within the same service area. We reviewed the 1998 awards and found that, generally, 
beds were awarded in accordance with the stated selection criteria. However, where there 
were exceptions, the Ministry was not able to locate the supporting documentation in all 
cases. 


In the 1999 process, in addition to the selection criteria used in 1998, the Ministry reserved 
the right to decline awards where there would be an unacceptable concentration of 
ownership of long-term care beds in the province or where applicants who were successful 
in four or more proposals could not demonstrate the financial and organizational capacity to 
develop that many sites. Based on our review of available documentation, successful 
applicants were selected in accordance with the stated selection process. In all cases where 
the highest scorer was not selected, there were multiple applicants who were perceived not 
to have the organizational or financial capacity to develop all of the beds that they could be 
awarded. External consultants hired by the Ministry performed the multiple applicant 


analyses and provided advice and recommendations to the Ministry. 


A new evaluation process was used in 2001 and the Redevelopment Project office hired 
external consultants to evaluate the proposals. The selection of successful applicants was not 
based on the highest ranked proposals. Instead, scores were used to group applicants within 
service areas into three “bands”. The first band included the top-ranked applicants. As one 
would expect, virtually all applicants in the first band received awards. The second band 
included all other applicants who scored over 65%: The applicants within this band were 
considered equivalent in terms of the quality and merits of their applications. Preference was 
given to applicants adding new beds to structurally non-compliant facilities and to 
applicants who proposed to build in preferred locations within service areas. As was the case 
in 1999, the Ministry reserved the right to consider the financial and organizational 
capacity of applicants who were successful in a number of proposals. The third band was 
made up of applicants who scored less than 65%, and, therefore, were not eligible for 
selection. 


According to the Ministry, approximately 900 new beds were awarded outside the regular 
processes to: 


* comply with recommendations pertaining to hospitals that were made by the Health 
Services Restructuring Commission; and 


* improve the economic viability of successful applicants from earlier RFPs that requested 


additional beds. 


Subsequent to the 2001 process, competitive selection processes were instituted in three 
service areas to award beds that either had been returned or were not initially awarded 


a 
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because of insufficient interest in 2001. Unlike the other three processes, where proposals 
were scored and ranked, the proposals for two of the service areas were evaluated based on a 
comparison of the positive and negative aspects of the proposals. The process for the other 
service area was still ongoing when we completed our audit work in this area. 









Recommendation 









To help demonstrate that awards for new long-term-care beds are based ona 
fair and open process that is consistently and objectively applied, the Ministry 
should ensure that the justification for all decisions is properly documented. 





Ministry Response 


The Ministry will do its best to ensure proper documentation of all decisions. 


Structural Compliance 


In our 1995 Annual Report, we noted that 68 nursing homes with approximately 7,000 
beds were so deficient they required major renovations or complete reconstruction to meet 
existing minimum structural and environmental standards. The Ministry is addressing the 
need to upgrade certain facilities through the Long-Term Care Redevelopment Project, 
which includes funding for the renovation of substandard facilities containing 15,835 beds. 


In 1995, we also noted that there were no comparable regulatory standards for homes for 
the aged and that the Ministry had not assessed the structural and environmental adequacy 
of these facilities. We recommended that the Ministry establish a plan to replace facilities 
that cannot meet structural and environmental requirements and that homes for the aged 
be assessed for structural and environmental deficiencies. 


In April 1998, the Ministry introduced new standards that would be applicable to all long- 


term-care facilities. Ministry staff classified all facilities into one of four categories: 
¢ class A facilities that met 1998 design standards; 


¢ class B and C facilities that did not meet the 1998 standards but did meet the 1972 
regulated standards; and 


e class D facilities that met neither the 1998 nor the 1972 standards. 


These classifications were primarily based on the review of building plans on file with the 
Ministry and the results of structural compliance assessments of nursing homes completed in 
1984/85. However, ministry staff did not have building plans for municipal or charitable 
homes. Facilities were permitted to appeal their classifications. Ministry staff, using a 
standardized survey template, reassessed these facilities, which resulted in a change of 
classification for a number of facilities. 





Long-Term Care Facilities Activity 135 





VFM Section 3.04 | 





“VE Section 3.04 | 


LL eS SSS sles 


At the request of the Ontario Association of Non-Profit Homes and Services for Seniors, the 
Ministry hired a consultant to conduct on-site reviews of the approximately 320 facilities 
that did not appeal their classifications. While the consultant identified 32 facilities that 
should have had a different classification, the Ministry adjusted the classification of only 
three facilities. The other 29 included 10 facilities containing 1,515 beds that the 
consultant stated should be class D rather than class C facilities and, therefore, should be 


renovated. Such renovations would normally receive financial support from the Ministry. 


By the end of 2001, only 42% of class D facility operators had agreed to rebuild their 
facilities to meet 1998 design standards. These operators would be eligible for a per diem 
premium of up to $10.35 per bed for 20 years. In December 2001, the Ministry 
introduced a new program to encourage more operators of class D facilities to commit to 
renovation. Class D facility operators now could also: 


* retrofit the facility to meet 1998 core design criteria, allowing design flexibility to reduce 
costs and enable better use of existing sites and structures. Depending on the extent to 
which the design varied from 1998 standards, operators would be eligible for a per 
diem premium of between $7.00 and $10.35 per bed for 20 years; or 


* upgrade the facility to meet all the class C structural classification criteria or spend at 
least $3,500 per bed. The operator would then be entitled to the structural compliance 
premium of $1.00 per bed per day. 


In addition, these operators would be eligible for additional financial support to help with 
unavoidable costs unique to redevelopment and retrofit projects, such as temporary facility 
leasing costs, temporary storage and operational costs, and fixed operating costs. Eligibility 
would be assessed on a case-by-case basis. By the time we completed our audit, the Ministry 
had received commitments from all of the operators of class D facilities to rebuild, retrofit, 
or upgrade their facilities. 


Class A, B, and C facilities are paid a structural compliance per diem premium, intended as 
compensation for previously completed structural improvements. Higher amounts are paid 
to facilities that more closely meet the 1998 design standards and have not received grants 
from the province to construct their facilities. 


Per Diem Structural Compliance Premiums, 2002 





: Facility Premium for Premium for Premium for 
Classification _ Nursing Charitable and Upgraded Class D 
8 Homes ($) Municipal Homes ($) Facility ($) 


A 5.00 1.50 N/A 
B* 2.50 hes N/A 
Ee 1.00 0.50 1.00 


* According to ministry policy, class B and class C facilities are not eligible for 
redevelopment funding and cannot improve their classifications if order to receive a 
higher premium even if they pay for their own upgrades. 




















Source of data: Ministry of Health and Long-Term Care 
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With respect to structural compliance premiums, we noted the following: 


¢ Because the province had funded 50% of the original construction costs of charitable 
and municipal homes, per diem premiums for these facilities were apparently reduced 
by 50%; but premiums were not reduced for nursing homes that, prior to 1996, 
received provincial funding through debt servicing and compliance premiums to 
partially compensate them for the cost of construction. 


¢ Ministry staff could not explain the apparent inequity whereby class A charitable and 
municipal homes receive only 30% of the premiums received by nursing homes 
whereas class B and C facilities receive 50%. 


¢ All class D facilities that upgrade to class C were entitled to the full $1.00 premium, 
even though the majority of class D facilities were either charitable or municipal homes 
and would otherwise only be entitled to a premium of $.50. The Ministry could not 


explain the basis for this premium. 








Recommendation : : _ © 


To help ensure that funding f io structural compliance is fair and to encourage 
facilities to meet the new design standards, the Ministry should: 





e ensure all facilities are properly classified; 


° review the structural compliance premiums to ensure that they are 
equitable and are achieving their intent; and 


e consider providing incentives for facilities to upgrade their classifications. 
Ministry essonse 
The Ministry is currently conducting policy work on asset management and 


facility renewal. This policy work will give consideration to the Provincial 
Auditor’s recommendations. 








PERFORMANCE MEASURES 


According to the Management Board of Cabinet’s minutes for May 17, 2000, “the 
Ministry needs to establish acceptable outcome measures and suitable commitments to 
better reflect its major activities/programs and contribution to core business outcomes.” For 
2000/01, the only performance measure in the Ministry's Business Plan for long-term-care 
facilities was to increase the number of long-term-care beds. 


The 2001/02 Business Plan stated that the goal of the Integrated Health Care Programs 
Division was to ensure that “the needs of Ontario’s changing population are anticipated and 
appropriate services and technology are available throughout people’s lives, close to their 
homes in their chosen communities” and included the following performance measures: 
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Residents of long-term-care facilities receiving quality care programs and services in a 
safe and home-like environment in accordance with provincial legislation, standards, and 
policies as shown by: 


* results of annual compliance reviews—100% of facilities inspected annually; and 
° redevelopment of existing [class D facilities] so that they meet or exceed current design 
standards. 
However, the Business Plan indicated no commitments or measures to address the 
appropriateness of services provided, such as the quality of care received by residents, or to 
determine whether programs offered by the facilities were meeting the needs of the 
residents. 


Senior ministry officials advised us that they recognize there are deficiencies in their 
assessment of program efficiency and effectiveness and are considering the implementation 
of the Resident Assessment Instrument (RAI). The RAT is currently used by all U.S. 
government-sponsored nursing homes as a uniform, standardized, comprehensive 
assessment instrument for all long-term-care residents in an effort to address concerns about 
basic levels of care. The RAI consists of a Minimum Data Set (MDS) and a series of resident 
assessment protocols describing how to use the MDS data for care planning. Quality 
indicators can be derived from the RAI to help identify potential problem areas for further 
review and investigation and may enable users to compare care among facilities. These 
indicators help identify the prevalence of inappropriate conditions. Ministry staff informed 
us that its level-of-care classification system currently collects about 60 pieces of data on each 
resident. Using MDS, over 400 pieces of information could be collected for each resident. 
Ministry staff believe that MDS would provide information that could be used to develop 
benchmarks and identify quality improvements. 


In October 2000, a review committee established by the Ministry recommended that the 
Ministry pilot the implementation of RAI at 40 facilities and conduct an evaluation prior to 
full implementation. The committee also recommended that the Ministry develop a 
funding methodology based on MDS and explore a partnership with the home-care sector 
to develop a common assessment tool. However, at the time of our audit, no progress had 
been made in implementing a pilot project. 








Recommendation 


To provide better accountability to the public and to help ensure that services 
of long-term-care facilities are provided efficiently and effectively, the Ministry 
should: 


* establish program goals, performance measures, and benchmarks and use 
them to assess performance; 

e take corrective action where necessary; and 

° report publicly on performance achieved. 
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Ministry Response 


Comprehensive annual reports from the long-term-care facilities’ compliance 
reviews are posted at the facilities for the public to see and are available to 
interested parties. 


It is the Ministry’s intention to move to a system that: 


¢ collects data and organizes the information to develop individualized 
resident care plans (this will reduce nursing time spent documenting and 
increase time spent providing care); 


e uses the data to develop risk-weighted quality indicators (quality indicators 
will enable benchmarks to be set that facilities must meet); and 

¢ compares facilities using the benchmark data. The Ministry will take 
corrective action where necessary. 





When the new system is fully implemented, the Ministry will make available as 
much information as possible on comparative indicators. 
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MANAGEMENT BOARD SECRETARIAT 


3.05—Electronic 
service Delivery 





BACKGROUND 


Over the past decade, many governments, including Ontario, have increasingly used 
electronic means to provide information about government services to individuals and 
businesses, as well as to deliver some of those services. This method of providing services 
is known as electronic service delivery (ESD). 


Public demand for government services delivered electronically is strong and growing. A 
recent study indicates that 75% of Canadians have access to the Internet at home, school, 
or the office. More than 80% of respondents to a recent private-sector survey of 
Canadian e-commerce users indicated they wanted to do such things as renew their 
driver's licence, complete government forms, and search for government program 
information on-line. In fact, 44% of survey respondents indicated that access to 
government services was one of their top six reasons for using the Internet—almost as 
many as those citing on-line banking and investing, 


In February 1998, the Management Board of Cabinet approved a new information and 
information technology (I&IT) strategy that recognized I&IT as a key enabler in 
improving the delivery of public services. Partly in response to increased public demand 
for ESD, this strategy included a proactive move towards “e-government” in Ontario. 
E-government is the comprehensive application of I&IT to the workings of government. 


Ontario’s vision for e-government encompasses various components, including the 
streamlining and automating of processes whereby government decisions are made and 
programs are delivered; the improvement of large public-sector programs in the health, 
justice, resource Management, transportation, and education sectors; and the facilitating 
of two-way citizen engagement or “digital democracy”. However, the component of 
e-government that affects the public most directly is ESD. Through ESD, the 
government is organizing and integrating services through such electronic means as call 
centres, interactive voice response systems, Web sites, e-mail, fax, CD-ROM, public 
access terminals and kiosks, and electronic payment systems. 


Currently, a number of Ontario government programs are provided through electronic 
means. For instance: 


SS 
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¢ Through approximately 70 kiosks installed at various locations throughout the province, 
individuals can obtain renewal stickers for their vehicle licence plates, change their 
address for health and transportation programs, and order fishing and hunting licences. 


¢ Through approximately 140 self-help workstations located in customer service centres, 
businesses and individuals can conduct business-name searches, register their business 
names, apply for retail sales tax vendor permits, and set up accounts for employer health 


tax and with the Workplace Safety and Insurance Board. 


¢ Via Internet sites, individuals can access information about government services offered; 
order publications; request birth, death, or marriage forms; file articles of incorporation 
and submit annual corporate information returns; apply for student loans; reserve 
camping park space; check traffic conditions; or access Ontario legislation. 


The government Internet site (www.gov.on.ca)—which was launched in 1995 to provide 
comprehensive information about government programs and to act as a central portal for 
citizens to access government on-line services—was redesigned in the fall of 2001 and 
receives some 500,000 visits per month. 


To accelerate the move to ESD, in June 2000, the Management Board of Cabinet 
approved a government-wide ESD strategy (titled “The Corporate Strategy for Direct 
Electronic Service Delivery”). The ESD strategy is aimed at improving the quality of 
service delivery to Ontarians and businesses by providing client-focused, integrated, 
accessible, and cost-effective government services electronically. In a September 2000 
speech, the Chair of the Management Board of Cabinet announced the government’s 
ESD commitment to: “By 2003, increase Ontarians’ satisfaction with government 
services by becoming a world leader in delivering services on-line.” The government 
further plans by that time to have over 80% of the services it delivers available to 
individuals through electronic means. The greatest growth area is expected to be in the 
provision of more services through Web sites. 


Management Board Secretariat (MBS) is responsible for the implementation of the 
government's overall I&IT strategy. Within MBS, the strategy for ESD is the 
responsibility of the E-government Branch (Branch) of the Office of the Corporate Chief 


Information Officer. 


The Management Board of Cabinet approved a performance management framework to 
track progress in fulfilling the government’s ESD strategy. The framework includes three 
elements and related commitments for 2001/02 and 2003, as illustrated in the following 


table. 
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Government’s ESD Commitments for 2001/02 and 2003 


Commitment for 2003 


75% of survey respondents 
would be satisfied with 
services delivered 
electronically. 




































Commitment for 2001/02 _ 


50% of survey respondents 
would be satisfied with 
services delivered 
electronically. 


Target Element 


Increasing Ontarians’ 
satisfaction with government 
services 























Ontario would be recognized 
consistently in the top five 
jurisdictions in the world for its 
ESD services. 


Ontario would be recognized 
consistently in the top 10 
jurisdictions in the world for its 
ESD services. 


Becoming a world leader in 
delivering services 
electronically 










75% of new ESD projects 
would be “leveraging” a 
common I&IT infrastructure. 


100% of new ESD projects 
would be “leveraging” a 
common I&IT infrastructure. 


Providing better government 
by improving public service 
efficiency 





Source of data: Management Board Secretariat 


While individual ministries continue to be responsible for the delivery of specific ESD 
initiatives related to their program areas, a number of ESD committees with ministerial 
participation provide leadership for the government’s overall ESD effort and ensure 
co-ordination of activities across participating ministries: 


¢ The Information and Information Technology Directions Committee oversees the 
government's [&IT agenda and provides strategic leadership for e-government 
initiatives. Chairing this committee is the Secretary of the Management Board of 
Cabinet, with members representing the Deputy Minister level throughout the 
government. 


* The Information and Technology Executive Leadership Committee works to ensure 
that Ontario makes the most appropriate I&IT investments. Chairing the committee 
is the Corporate Chief Information Officer with members representing the Chief 
Information Officer level from the corporate organization and the I&IT clusters. 


* The e-Government Leadership Committee strategically co-ordinates the 
e-government initiative. The Corporate Chief Information Officer chairs the 
committee with membership representing mainly the Assistant Deputy Minister level. 


* The Chair’s Advisory Council on e-Government provides advice to the Chair of the 
Management Board of Cabinet on the transformation of government through the use 
of I&IT. The Corporate Chief Information Officer is the chair and membership is 


from across the public and private sectors. 


AUDIT OBJECTIVES AND SCOPE 


The objectives of our audit of electronic service delivery were to assess the extent to which: 
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¢ Management Board Secretariat (MBS) has systems and procedures in place to 
continuously monitor, measure and report on the government’s progress towards 
meeting its objective of increasing customer satisfaction by becoming a world leader 
in the provision of electronic services by 2003; and 


* ministries are developing and delivering electronic services in accordance with best 
practices and with due regard for economy and efficiency. 


We identified audit criteria that would be used to address our audit objectives. These were 
reviewed and accepted by senior officials at MBS and further reviewed with the ministries 
visited. Our audit work covered the period to March 31, 2002. 


The scope of our audit included discussions with staff and a review and analysis of 
relevant policies, procedures, and related documents at MBS and at the ministries of 
Consumer and Business Services; ‘Training, Colleges and Universities; and 
Transportation—three of the ministries currently delivering major ESD projects. 


Our audit was performed in accordance with the standards for assurance engagements, 
encompassing value for money and compliance, established by the Canadian Institute of 
Chartered Accountants, and accordingly included such tests and other procedures as we 
considered necessary in the circumstances. 


With the exception of work done by Internal Audit Services at the Ministry of Consumer 
and Business Services, which assessed the evaluation process used for one significant 
consulting contract, we did not rely on the work of internal audit because there had been 
no recent reports on ESD projects. 


OVERALL AUDIT CONCLUSIONS 


The government has set ambitious targets for ESD, and while it has made significant 
strides in implementing ESD to date, we concluded that it will likely fall short of meeting 
its ESD implementation targets. In our view, a number of issues need to be addressed to 
accelerate the pace of ESD implementation and to ensure that ESD investments provide 
value for money. As well, a more proactive and hands-on central management of the ESD 
initiative is needed. For example: 


¢ Ministry quarterly reports on the delivery of ESD projects showed that 52% of 
ministries’ ESD projects were behind target in June 2001, in that they had either not 
yet been initiated as planned or had been delayed or deferred in some manner. By 
December 2001 even fewer ESD projects were on target. 


¢ The E-government Branch’s ability to conduct meaningful analysis of the current status 
of projects vis-a-vis those originally planned was impaired. Projects had been dropped, 
delayed, deferred, redefined, or combined in a myriad of ways, which can happen for 
valid reasons. However, the Branch did not have sufficient documentation of the 
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reasons for many of these changes. In addition, ministry quarterly reports were being 


provided six months behind schedule. 


* The Branch is responsible for the government-wide ESD plan, but its authority to 
deliver on this plan is limited to providing senior management with analysis of 
ministry progress to date and making recommendations as to where corrective action 
or additional resources may be needed. However, even in this capacity, the Branch’s 
reporting to senior management to date has not been timely and has been limited and 
lacking in recommendations for future action. 


As well, while it originally intended to provide the Chair of Management Board of 
Cabinet with a progress report in spring or summer 2001, at the time of our audit, 


the Branch had not yet provided this report. 
* The funding needed to deliver ESD projects was not addressed when the ESD 


strategy was approved. Lack of resources was the reason most often cited by ministries 
for their inability to deliver on planned projects. 


* The Branch has set 2001/02 and 2003 performance targets for customer satisfaction, 
world leadership, and ESD project leveraging; and while a recent survey of customers 
using existing ESD services indicates that the 2001/02 customer satisfaction targets are 
being achieved, the Branch does not have conclusive evidence that it is Meeting its goal 
of Ontario being among the world’s 10 best jurisdictions in delivering electronic 
services, nor is there evidence that ESD projects are integrated, leverage a common 
I&IT infrastructure, and incorporate common components. 


In addition, ESD performance measurement efforts to date have been poorly co- 
ordinated between the Branch and the ministries delivering ESD programs, and 
operational or efficiency improvement measures, or assessments of the economic costs 
and benefits of ESD projects, remain to be developed. 


* Communications efforts to promote ESD have been insufficient to increase public 
awareness and usage of the services delivered electronically. Usage of some ESD 
services are significantly below target levels. 


We reviewed four high-impact service-delivery projects at the ministries visited and noted 
that, while the ministries had implemented a number of good project management 
practices on these priority projects, with respect to security practices and service 


availability, there was some room for improvement. 





Overall Management Board Response 

Ontario has embarked on an ambitious journey towards electronic service 
delivery (ESD) in the context of its overall e-government agenda. 
Management Board Secretariat (MBS), in co-operation with ministries and 
| other partners, is moving forward simultaneously in a number of areas to | 
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increase satisfaction with government services by becoming a world leader in 
delivering services electronically. 


We look to this audit as a means of identifying opportunities to employ best 
practices to accelerate the pace of ESD implementation and ensure that the 
government meets its 2003 target. MBS and the three line ministries involved in 
the audit have already taken action on many of the Provincial Auditor’s 
recommendations and are satisfied that we will achieve our 2003 target. 


We are pleased that the audit report acknowledges that significant strides have 
been made and appreciate the advice that the Provincial Auditor has provided 
as we undertake this large-scale transformation. 














DETAILED AUDIT OBSERVATIONS 
PROGRESS REPORTING 


As indicated earlier, the government’s ESD strategy is directed towards achieving stated 
targets by 2003. With a view to meeting these targets, all ministries submitted ESD 
action plans in fall 2000 and were to subsequently report quarterly on their progress 
against these plans. The ministry action plans set out planned improvements over the 
two-and-a-half year period from September 2000 to April 2003. The plans formed the 
basis of MBS’s corporate ESD plan (titled “Enterprise ESD Strategy and Action Plan”), 
which was approved by the Management Board of Cabinet in February 2001. MBS’s 
analysis at that time indicated that 58% of government services were then available 
through at least one electronic mean and that if all planned service improvements were 


executed this percentage would grow to 83% by 2003. 


Ministry Reports 


While considerable work went into the preparation of ministry action plans and the 
overall corporate ESD plan, the ministry quarterly progress reports that had been received 
at the time of our audit indicated that actual progress against these plans has been 
disappointing and that the inventory of projects being tracked has dropped significantly 
without an explanation for this drop. 


Ministries were experiencing significant difficulties in completing their ESD projects. 
Specifically, the first reports indicated that by June 2001, 52% of projects reported on had 
either not been initiated as planned or were behind schedule vis-a-vis the timeframes 
envisaged in the original plans. The second reports indicate even further deterioration in 


progress as of December 2001. 
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We also noted that the project information in these reports did not correspond with that 
reported on in the original project plans. The reports had not been assessed with respect to 
the drop in the number of projects being reported on. While discussion with staff at the 
Branch and our own analysis indicated that many of these adjustments relate to the 
combining of projects that were originally considered separate initiatives—and some would 
appear to be due to successfully completed projects that are no longer being reported—in 
many cases it would appear to be the result of planned projects simply being dropped from 
the reports with no reasons being provided. 


Although meeting ESD project commitments is the responsibility of the applicable Deputy 
Minister, we also noted that MBS had not made any formal inquiries to the ministries as to 
what corrective actions, if any, had been developed to deal with any of the delayed projects. 


Reporting Cycle 

Given the ambitious ESD performance targets for 2003, the ability to monitor project 
progress on a timely basis is critical. However, the reporting cycle has been subject to 
considerable slippage, which has contributed to the difficulty in pro-actively addressing 
project delays. Initially, ministries were to submit their first quarterly report in April 2001. 
However, this initial reporting requirement was pushed back twice such that it was not 
until the fall of 2001 that ministries began submitting their first reports. 


Many of these initial reports were found to be inadequate, and the Branch continued 
working with ministries throughout the fall and winter of 2001 to clarify their 
information requirements and ensure submitted reports contained all the necessary data. 
As of March 2002, two ministries had still not submitted their first quarterly reports. 


Because of the difficulties in completing the first quarterly report, the original deadline 
for the second quarterly report was abandoned and a due date of March 5, 2002 was set 
for the second report. So, while it was anticipated that ministry reports would be prepared 
every three months, in the span of almost a year (starting from the originally scheduled 
due date of April 2001 for the first report), only two quarterly reports were submitted. 


The Branch is now considering changing the requirement for future reports such that they 
would have to be submitted only every six months rather than every three months. While 
this may be more convenient for ministries, it will certainly impair the Branch’s ability to 
learn of and react on a timely basis to any project slippages or other problems. 


Reporting of Progress to Senior Management 


The reporting difficulties outlined above have contributed to a further delay in providing 
senior management at MBS and at ESD ministries with meaningful summary analysis of 
progress to date. For instance: 
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* Originally, the Branch planned to provide a summary of ministry progress to the Chair 
of Management Board in the spring or summer of 2001, shortly after receiving the first 
ministry reports. At the end of our audit, in March 2002, only a draft of this progress 
report had been prepared, and it had yet to be finalized or submitted to the Chair. 
Furthermore, this draft contained status information only, with no recommendations for 
action or measures to deal with any of the delayed projects. 


¢ Although results of the first draft progress report were presented in December of 2001 
to the e-Government Leadership Committee, the committee itself had difficulty 
assessing the results and indicated that more analysis was needed. 


¢ At the end of our audit, the Branch had not as yet produced any analysis or report on 
the results of the second ministry quarterly reports. 


Without ongoing and timely review of progress to date, corrective action cannot be 
initiated at the senior level. Given that the Branch has no effective means to deliver on the 
government's mandate but must work through the ministries actually responsible for the 
provision of ESD services, such reporting is one of the few tools it has available to push 
the ESD initiative forward. That is, while the Branch does not have the power to enforce 
or fund ministry initiatives, it is poised to assess and communicate progress and problems 
to committees and senior ministry management, which can in turn initiate corrective 
action. However, as outlined above, the Branch has provided little progress reporting to 
date. 





Recommendation _ 


To ensure that ministry progress in completing improvement projects for 
electronic service delivery (ESD) is adequately assessed and timely 
corrective action is initiated where appropriate, Management Board 

Sect tara! should: 





e ‘require that all ministries submit their required eg on time and 
formally follow up when they fail to do so; 


¢ track the service improvements identified in the original ministry ESD 
plans and compare them to expected and actual results so that a 
complete assessment of ESD ac pene vis-a-vis original 
targets can be made; © 

¢ consider initiating formal follow-up procedures and asking ministries 
who are significantly behind target to develop corrective action plans; and 

e analyze all submitted reports and provide a summary analysis with 
recommendations on a timely basis to the Chair of Management Board 
of Cabinet and appropriate ESD advisory committees. 
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Based on the Provincial Auditor’s findings and our own consultations with 
ministries, Management Board Secretariat (MBS) is reviewing the frequency 
of and approach to ministry ESD progress reporting to ensure we have 
timely notice of issues and can take steps centrally to remove horizontal 
barriers. We will streamline the reporting process to focus on meaningful 
indicators and critical deliverables. The Branch will escalate delays in 
progress reporting to the Deputy Minister level and identify significant 
changes in ministry ESD plans to the Chair of the Management Board of 
Cabinet. Cabinet Office has also established performance plan 
commitments for Deputy Ministers as a means of monitoring achievement 
of ministry ESD commitments. A report was provided to the Chair of the 
Management Board of Cabinet in July 2002 and covered the ministries’ first 
and second progress reports. 


Management Board Response 


It is important that ministries continuously evolve and refine their ESD 
plans. Therefore progress towards the strategic ESD target cannot be 
derived simply by tracking the progress of the initial set of projects 
included in plans submitted in October 2000. 


For the 2003/04 planning year and beyond, in addition to tracking progress 
on critical deliverables, MBS will also assess ministry progress in terms of 
the approved performance measures. Assessing actual performance 
outcomes against key performance measures will provide us with a direct 
indication of progress towards the target of increasing client satisfaction. 
In addition, frequent assessment of critical ministry projects against 
planned milestones will serve as an early warning signal to MBS regarding 
ESD progress. 
bs 











THE FUNDING OF INITIATIVES 


Unlike other government initiatives, where typically the initiative and its required funding 
are approved concurrently, the government’s planned ESD initiative was approved 
without addressing the funding needed to support it. In fact, in our review of the Branch’s 
development of the government’s ESD strategy, we noted little consideration of funding 
needs, resource planning, or estimation of total expected ESD costs. We were informed 
by branch management that a separate funding envelope for ESD was not considered: 
funding for ESD would be assessed and assigned in the context of overall business 
practices. 


The separation of the strategy from its funding requirements concerned members of ESD 
committees. For example, minutes of a May 2000 meeting of the I&IT Directions 
Committee reflect members’ concerns about the lack of funding details. In reviewing the 
upcoming submission to the Management Board of Cabinet for approval of the ESD 
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strategy, the committee indicated there was a need to provide estimates of the financial 
requirements of the strategy decision to the Management Board of Cabinet. 


Despite such concerns, this work was not completed at the time the ESD strategy was 
presented to and approved by the Management Board of Cabinet in June 2000. In fact, 
the first serious attempt to identify ESD funding requirements only occurred in fall 2000 
when ministries prepared ESD action plans. The ESD strategy and 2003 targets were thus 
approved in June 2000 without full knowledge of what funding would be required to meet 
them. 


The October 2000 ministry action plans identified 588 service areas and outlined funding 
requests for electronic improvements to most of these services—requests for $66 million for 
2001/02, and $187 million over the four-year period to 2005. As mentioned earlier, these 
plans formed the basis of the corporate ESD plan approved by the Management Board of 
Cabinet in February 2001. However, consistent with the view that funding would be dealt 
with as part of the regular business planning process, the Management Board of Cabinet 
approved the corporate plan but did not approve the funding needed to execute it. 


There has been one exception to the overall funding approach for ESD initiatives when 
ministries were afforded the opportunity to request central, dedicated funding. In 

April 2001, after reviewing 135 requests, the analysis and recommendations provided by 
the Branch to Cabinet resulted in central, dedicated funding of $15.2 million for 14 ESD 
projects for the 2001/02 fiscal year. 


Our review of ESD committee minutes indicated that members continued to raise 
funding concerns, with doubts being expressed about the ability to meet the government’s 
ESD goals under existing funding conditions. With respect to the 14 ESD projects 
discussed above, we noted that members of the [&IT Directions Committee expressed 
concern about the relatively low number of initiatives going to the Management Board of 
Cabinet for approval of specific ESD funding. While the minutes further indicated that a 
review of the business cases for other projects was underway, which might lead to further 
specific funding requests, we noted that at the time of our audit there had been no other 


centrally led ESD funding approvals. 


We noted that, in contrast to the Ontario government, the federal government, in its 
2000 Budget, specifically committed $160 million over two years to design and launch its 
on-line strategy. The federal government committed an additional $600 million in its 
2001 Budget over the following four years to complete the implementation of its on-line 
strategy by 2005. This funding commitment may have contributed to the federal 
government's current success in international rankings of on-line services. For instance, 
Accenture—a major international consulting firm—has ranked the Canadian federal 
government number one in the world for the last two years in the provision of on-line 
services and in the area of e-government leadership. 
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In Ontario, our view is that the lack of an overall funding commitment in support of the 
approved ESD strategy has contributed to the current low rate of progress vis-a-vis 
ministry action plans. We noted that, at the time of our audit, 13 of the 14 (93%) ESD 
projects that obtained specific funding were on target for successful project completion. 
This contrasts sharply with the much lower, current on-target rate for remaining ESD 
projects. In fact, we noted that in ministry progress reports, where a rationale is provided 
to account for delays, the lack of resources is a prime reason cited. 


We noted that the Chair’s Advisory Council on e-Government also had concerns about 
the government's approach to funding ESD initiatives. At its second meeting, the 
Council discussed current funding mechanisms—where estimates and funding 
allocations are determined on a program-by-program basis—and noted that they are not 
suited to large, cross-jurisdictional IT initiatives, which are integrative in nature rather 
than program specific. 


Given the current constrained funding environment and the large number of projects 
that are behind schedule, we believe it is essential that ESD projects be prioritized on a 
government-wide basis to ensure the most strategic and value-added ESD initiatives from 
the public’s perspective are identified. Under the current approach, the risk is lower- 
priority ESD projects may get funded over higher-priority projects depending on the 
availability of funds within each ministry. 





Recommendation | 


To ensure appropriate funding of electronic service delivery (ESD) 
initiatives, Management Board Secretariat should: 


e review the current funding mechanisms for ESD initiatives to determine 
if alternatives to the current funding model should be considered; 

* ensure funding provided is directed at the most strategic initiatives 
from a government-wide perspective; and 

* consider developing a proposal to centrally fund the delayed ESD 
projects that are most critical to improving program delivery. 





Management Board Response 


When the ESD Strategy was developed, various funding models were 
considered, and Management Board Secretariat (MBS) chose not to 
establish a central fund for ESD on the basis that, to the greatest extent 
Possible, decisions on ESD should be taken in the context of all business 
priorities. In addition to the 14 initiatives funded through the 2001/02 
business planning process, the Management Board of Cabinet approved in 
May 2002 $16 million more for 19 strategic ESD projects for the 2002/03 
fiscal year. MBS will again review funding requests for ESD projects as part 
of the 2003/04 planning and allocations cycle. MBS is committed to 
exploring alternate funding models, and monitoring the effectiveness of 

| models in use in other jurisdictions. 
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PERFORMANCE MEASUREMENT 


As mentioned earlier, the Branch’s formal performance measurement framework for ESD 
currently incorporates three target elements and, to its credit, specific commitments to 
measure progress towards the achievement of these targets (see table on page 142). The 
current targets relate to customer satisfaction, world leadership status, and the 
“leveraging” of a common I&IT infrastructure. As part of our review of ESD 
performance measurement, we assessed progress to date on meeting these targets. 


Customer Satisfaction 
In early 2002, the Branch, in the context of the framework approved by MBC, 


commissioned a comprehensive customer service satisfaction survey that measured, 
among other things, the satisfaction levels of over 1,000 recent users of government ESD 
services. Input was sought from both business clients and the general public. The results 
indicated that current satisfaction levels with Ontario’s ESD services were well over the 
targeted 50% threshold. Generally, users who accessed Ontario programs via electronic 
means were happy with the service they received. 


We concluded that the Branch had sufficient evidence to demonstrate that it had 
exceeded its 2001/02 commitment for customer satisfaction. 


World Leadership Status 


Ontario has taken great strides over the last several years in establishing and further 
developing a number of key ESD initiatives. As well, the Branch has been instrumental in 
developing legislation and policies that will support ESD by providing needed 
frameworks—for instance, for security and privacy requirements. Indeed, our audit work, 
and the work of others, indicate that Ontario currently offers a world-class selection of 
electronic services and has established a solid framework to support them. 


However, we do not believe that the Branch has clearly demonstrated that Ontario has 
met its 2001/02 target of being among the top 10 ESD service providers in the world. 
While a number of studies have favourably compared the federal government's on-line 
offerings with those of other international jurisdictions, the Ontario government cannot, 
in our view, infer achievement of its targets from such federal benchmarking studies. 


In the course of our work, we examined comparative studies and performed our own 
comparisons of on-line ESD services offered in Ontario and other Canadian provinces, as 
well as selected international jurisdictions. It is important to note that studies to date have 
only dealt with on-line services. ESD includes many other electronic means, including 
kiosks, public self-help workstations, call centres, and interactive voice response systems. 
We noted that Ontario has invested significantly in all of these technologies as part of its 
ESD strategy, but because of insufficient data from other jurisdictions, we limited our 
comparative assessment to on-line offerings. 
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A 2001 government-funded ESD study of all provincial jurisdictions in Canada ranked 
Ontario second behind British Columbia. Our own comparison of four provinces 
indicated that Ontario compared favourably with these jurisdictions, but also ranked 
second behind British Columbia, primarily because of the much greater number of on- 
line services BC offered. Our assessment took into account the number, breadth, and 
depth of services offered; transactional capabilities; and user friendliness. 


However, Ontario's goal is to be among the top 10 (by 2001/02) and then top 5 (by 
2003) jurisdictions, not in Canada, but in the world. In our view, many other 
international jurisdictions currently offer on-line services that are superior to Ontario’s. In 
an annual comparative study of government on-line services by the consulting firm 
Accenture, Hong Kong was ranked tenth worldwide in 2001 and Ireland was ranked 
tenth in 2002. We therefore compared on-line services offered in Ontario to those offered 
in these and two other jurisdictions that had garnered recognition for the quality of their 
on-line services. While cross-border comparisons are more difficult than interprovincial 
comparisons because of the global differences in government structures and mandates, we 
focussed on 18 public services common to all jurisdictions, including birth and death 
certification, student assistance, driver licensing, vehicle/ plate renewal, and business 
registration services. We ranked Ontario fourth out of five in this assessment, behind both 
Hong Kong and Ireland. Extrapolating from this additional data, it is questionable 
whether Ontario is currently in the top 10 worldwide in the delivery of on-line services. 


In terms of Ontario’s potential for improvement, we noted that the government-funded 
consulting study found that of Ontario’s 467 on-line services, 60% were simply 
informational and only 6% had transactional applications available. In this respect, the 
study further indicated that 86% (or over 170) of Ontario government sites that had the 
potential to provide transactional services had not yet done so. This would indicate that 
in view of enhancing its world leadership status, there is still considerable room for 
Ontario to improve the electronic delivery of its services. 





Recommendation 


To improve the performance of electronic service delivery (ESD), 
Management Board Secretariat should: 


° expand current benchmarking exercises to include more types of 
electronic service delivery; and 


* use and disseminate the results of benchmarking studies to help 
ministries identify areas needing improvement and develop action plans 
to implement the required improvements. 


Management Board Response 


Given the growing number of benchmarking studies arid the variable 
methodologies of these reports, Management Board Secretariat (MBS) 
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acknowledges that conclusively demonstrating Ontario’s world leadership a 
status is, and will continue to be, a challenge. As the Provincial Auditor 
recommends, MBS is using results of recent global benchmarks to identify 
areas where improvements are needed to sustain our global leadership. 
Client satisfaction surveys and organizations that lead benchmark studies 
note five key criteria that influence satisfaction, and ultimately influence 
where governments are ranked internationally. We have taken steps to 
improve in each of these areas: vision and implementation, customer- 
centric approach, customer relationship management, volume and 
complexity of e-services, and customer centric portals. A study released in 
Germany in June 2002, for example, placed Ontario third amongst 60 
jurisdictions surveyed. Two Ontario projects received awards at the E-Gov 
2002 Exposition in Washington, D.C. 

















Leveraging and Integration 


We found no convincing evidence that 75% of Ontario’s ESD projects are “leveraging” a 
common I[&IT infrastructure. In fact, no clear definition has been developed as to what 
constitutes a “leveraged” project, and no assessments have been made as to how many 
current ESD initiatives could be considered “leveraged”. Branch documentation indicates 
that achievement against this target will be assessed by “internal metrics’—none of which 
have been currently defined. We conclude therefore that this measure is at present too 


vague to be considered useful. 


The issue of ESD project “leveraging” is, in our view, part of a much larger and 
important issue, that of ESD integration. Ontario’s long-term vision for ESD services is 
that they will be seamlessly integrated within the overall context of government program 
delivery. Working towards this goal has many aspects, some of which are discussed below. 


In looking at the development of a common I&IT infrastructure, we reviewed the 
government's 1998 I&IT strategy and noted that it incorporated a three-year action plan 
designed to address, for the year 2001 and beyond, a number of core infrastructure issues 
to transform the government’s IT environment. These changes were considered critical to 
providing a foundation for supporting the government’s major service delivery and 
restructuring projects, including its future ESD initiatives. However, at the time of our 
audit, much of the work contemplated under this strategy has not been completed or has 
been found to be unworkable as originally envisioned. For example: 


e The strategy incorporated a “target information architecture” for the sharing, where 
appropriate, of information pertaining to individuals, businesses, and land. This data 
model remains to be implemented. 


¢ The strategy envisioned a standardized desktop to be implemented throughout the 
government, with each ministry cluster choosing one suite of application products. 
This standardization has not been completed. 
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¢ The strategy envisioned a common help-desk facility to provide first-level solutions and a 
government-wide problem management system. These have not been established. 


Another key government-wide integration initiative that MBS started in 1999 was the “21 
common-component project”. The project’s purpose was to develop centrally approved 
architecture and applications for 21 general-purpose components that are common to 
many ESD services. General-purpose components would include a credit-card payment 
system, a shopping cart facility, and an address changing function. Once developed, these 
approved components could then be shared with all ministries, thus reducing duplication of 
effort and investment in developing new ESD components. As well, shared components 
would accelerate movement toward a common “look and feel” to ESD services being 
accessed by Ontarians, thereby minimizing user confusion or frustration arising from a lack 
of consistency. However, at the time of our audit, with the exception of one component (the 
on-line credit-card payment system), these 21 common components were still in the 
development stage. 


Another aspect of integration is what is known as system integration. In a seamless ESD 
application, the “front-end” service that is accessed by the general public through a Web 
site or other electronic means is connected to the “back-end” application system and 
database. For instance, a system that allows users to change their addresses on-line would 
be connected to the government database that houses and updates this information. This 
type of integration minimizes unnecessary data handling, edit checking, and transaction 
processing, which makes integrated systems more efficient and less prone to error. To 
date, few government ESD systems are fully integrated, a point also raised in private- 
sector studies. As one significant example, we noted that the Ministry of ‘Transportation’s 
on-line vehicle registration system has not as yet been integrated with the legacy Vehicle 
Registration System on the Ministry’s mainframe computer. As a result, ministry staff 
must retrieve all information submitted on-line, print this information out, and manually 
re-input this data into the legacy system. There is also no on-line validation to ensure data 
being entered by a user is valid and complete (for instance, that all required fields are filled 
in), adding to the administrative burden of processing and following up on these 
transactions. 


As mentioned earlier, a common “look and feel” is another way in which ESD services 
can be integrated. A common look and feel is implemented across an organization’s Web 
sites and its numerous linked pages to add to ease of use and is thought by experts to 
increase general confidence, comfort, and trust in an organization’s ESD offerings. Our 
review of Ontario’s numerous on-line offerings indicates the province still has a long way 
to go in this area. Ministry sites often differ significantly in both appearance and in 
transactional interfaces. 
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Recommendation 


To ensure that electronic service delivery (ESD) is integrated, Management 
Board Secretariat should: 


¢ clearly define the meaning of “leveraged” ESD initiatives and 
benchmark ESD projects against this target; 


¢ complete the development of a common information and information 
technology (I&IT) infrastructure; 


* complete the “21 common-component project” as soon as possible so 
that the efficiency gains and effectiveness of these components can be 
realized wherever feasible in existing and future ESD projects; 


e develop a strategy for system integration of legacy systems with the 
newer “front-end” Web server systems; and 


e develop a strategy to continually standardize ESD interfaces throughout 
the government to achieve a common “look and feel”. 


Management Board Response 


Management Board Secretariat (MBS) is working to refine the corporate 
ESD performance measure of better government through information and 
information technology (l&IT), (that is, the leveraging of a common I&IT 
infrastructure). Enterprise-wide common infrastructure (for example, 
network, directory, security) is extensively deployed and in use, as are 
initial ESD common components. The Office of the Corporate Chief 
Information Officer has recently confirmed the next infrastructure priorities 
for the coming few years, including a strategy to accelerate the 
development and use of the 21 common electronic service delivery 
components. We agree with the Provincial Auditor’s findings that common 
components will significantly reduce the development time for ESD 
systems and will also serve to contain ongoing operating costs. 


MBS agrees that integration of back-office legacy systems with Web server 
systems is an important aspect of system and service integration. It is also 
the most complex and risky step in the maturation of electronic service 
delivery, which is why we are approaching it with caution and careful 
consideration. Major legacy system renewal of the drivers and vehicles 
systems is underway at the Ministry of Transportation and will, among 
other factors, support front-end Web applications. 


The implementation of a common interface for clients is a priority. We are 
revising our “look and feel” Web standards to ensure we migrate to a 
common approach across the government, including ensuring that sites 
are accessible to persons with disabilities. MBS will ensure that ESD 
interface standards are given priority by the government's I&IT Standards 
Council and the GO Web Committee. 
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Other Performance Measures 


Given that governments have only recently been delivering services electronically, it is not 
surprising that ESD performance measurement systems—in Ontario and throughout the 
world—are essentially in their infancy. However, systems to estimate expected ESD 
performance and cost improvements and to measure actual performance against such 
targets need to be developed to support continued investment in ESD. 


As already outlined, the Branch’s performance-measurement efforts to date have primarily 
had an external focus (for instance, the impact on users and Ontario’s ranking against 
other jurisdictions). As well, MBS’s performance measures have largely been developed 
and implemented centrally, without ministry involvement. While we support the efforts 
made to date, we believe that to fully drive the ESD initiative forward more is needed— 
specifically, a mix of both internal and external performance measures and a more co- 
ordinated approach to measuring performance centrally and across ministries. 


Third-party studies and our own observations indicate that government business cases for 
ESD initiatives to date have focused almost exclusively on expected outcomes related to 
improved customer service. While service improvement is of course a critical factor, it 
alone is insufficient to measuring performance. Another critical consideration is cost 
effectiveness. 


There are a number of possible approaches to measuring cost effectiveness. For new ESD 
initiatives, business cases could compare projected application development costs against 
projections of economic and non-economic benefits. Economic benefits could include 
such elements as expected business growth; savings in terms of reduction in front-line 
staffing needs; cost reductions related to less document handling and error correction; 
reduced or eliminated data inputting and/or processing; and reduced printing, 
publication, or mailing costs. Non-economic benefits could include: improved customer 
service as measured by improved transaction turnaround times and greater customer 
satisfaction. After ESD implementation, measures should be used to validate business 
cases and support continued investment in ESD improvements. Such measures could 
include: the cost per transaction, statistics for user growth and repeat users, data accuracy 
and efficiency improvements, improvements in the timeliness of service, and analysis of 


user feedback. 


Our observations at the ministry level indicate that to date ESD performance 
measurement practices are quite basic and ad hoc in nature. For instance, while the 
Ministry of Transportation had established standards for its renewal service for licence 
plate validation stickers, actual performance was not being adequately monitored to 
determine whether they were being met and whether corrective action should be 
considered. At the Ministry of Training, Colleges and Universities, the existing 
performance measurement strategy needed to be updated to include more ESD 
performance targets and measures with respect to on-line student applications. 
Furthermore, the approaches applied across the government lacked consistency, and none 
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of the ministries we visited had integrated their measures with those used centrally by the 


Branch. 


Recognizing the need to develop sound and consistent approaches to ESD performance 


measurement, in fall 2001, the Branch launched a baseline performance data initiative. 


The initiative is to include a process for data collection and analysis through a liaison 


appointee from each ministry that is aimed at identifying gaps in performance measures 


with a view to closing such gaps. Through this initiative, the Branch will attempt to 
integrate ministry ESD performance measures with the current measures used by the 
Branch for customer satisfaction and world leadership status. At the time of our audit, 
this initiative was still in its early stages. We noted that the ESD staff we dealt with at 
ministries were largely unaware of it. 








Recommendation 


To ensure accurate and useful performance measurement of the 
government’s ESD initiatives, the Branch should: 


e develop additional approaches to ESD performance measures that 
include a mix of external and internal targets and improved 
business-case methodologies; and 


e work with ministries to help them develop performance measurement 
approaches in an integrated manner across program areas. 


Management Board Response 


Management Board Secretariat (MBS) welcomes the advice of the 
Provincial Auditor to strengthen the performance measures by expanding 
the range of targets set. Ontario has deliberately set a target that is client 
focused and outcome based. MBS will explore ways to further develop 





internal and external targets that meet the needs of ministries in measuring 


their own performances in the context of the corporate ESD measures. 


MBS has already taken steps to facilitate the development of better ESD 
project business cases, including the development of a tool kit for use by 
ministries. 


In December 2001, MBS set up an ESD Performance Measures Network of 
representatives from all ministries to: 


e facilitate the collection of ministry baseline data on ESD performance 
measures, 


¢ pilot and internalize ESD performance measurements developed by 
MBS, 


¢ share best practices in the area of performance measurement, and 
e drive the development of program-level ESD performance measures. 











: 
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We recognize the importance of ongoing dialogue with ministries to assist 
them in understanding the ESD performance-measurement framework, and 
we will continue to work with ministries to develop program-level 
indicators. For example, at Showcase 2002, MBS hosted a panel discussion 
on performance measurement to reinforce with attendees from all 
ministries and I&IT clusters the importance of measuring results and acting 
on performance gaps. We will continue to actively assist ministries to 
identify means of improving outcomes up to and beyond 2003. We are 
initiating the use of a “best in class” methodology for measuring 
organizational maturity in regards to client satisfaction to pinpoint specific 
areas that we must focus on to achieve our 2003 target. 

















PROMOTION AND COMMUNICATIONS 


‘To ensure the public is aware of the government's ever-growing number of electronic 
services and to promote their use, when the Management Board of Cabinet approved the 
ESD strategy in June 2000, it directed MBS to develop an ESD communications plan 
prior to the public announcement of the 2003 targets. In February 2001, when it 
approved the ESD corporate plan, the Management Board of Cabinet again directed 
MBS to complete this communications plan and report back with it in April 2001. 


An e-government communications plan was finally completed in the spring of 2001. The 
plan noted that, despite promotion efforts to date, media coverage of the government’s 
ESD commitments and activities had been negligible. Recognizing that increasing public 
awareness of available electronic services was a critical component of the ESD strategy, the 
plan envisaged high-profile fall 2001 launches for major ESD initiatives, which would be 
followed by several “large-impact” announcements for several months to build and 
sustain momentum. 


In our view, however, ESD promotion efforts throughout 2001 were relatively modest. 
The most significant public promotion appeared to be the fall Showcase event, which was 
a series of seminars and presentations that highlighted ESD project success stories. While 
this appeared to be an excellent forum, Showcase participants were primarily from within 
the public service, public institutions, or the wider community of IT and ESD vendors 
and consultants. It was not, in our view, an effective forum for disseminating ESD 
information to the general public—the actual target users of new ESD applications. 


The first major communication to the public occurred much later, in the government’s 
spring 2002 quarterly report to Ontario residents. This communication devoted several 
pages to the government's ESD initiatives, providing an excellent overview of currently 
available and upcoming electronic services and promoting the main government portal as 
an exploratory tool to access these services. We encourage this type of communication on 
an ongoing basis and believe continued promotions of this type have the best chance of 
reaching target audiences, building awareness, and enticing potential first-time users to 


try ESD. 
eee 
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In addition to needing better public communication, the promotion of ESD requires 
internal co-ordination across the government. To achieve the best results and to ensure 
consistent messaging, MBS and ESD ministries should work together. MBS’s ESD 
communications plan itself incorporates this view, envisaging a central strategy to be used 
by all ministries in providing common messages. We noted that more work is needed to 
ensure this co-ordination is achieved. Most ministry staff we dealt with were unaware of 
the central strategy. 


We also had some concerns with the specific ESD communication approaches used by 
the ministries we visited. At the Ministry of Transportation, since the implementation of 
the on-line renewal service for licence plate validation stickers in October 2000, there has 
been very limited promotion of this service to the public. This has contributed to a very 
low penetration rate for this service: 


¢ Only 4,257 vehicle licence renewals were processed on-line for the period from 
November 2000 to January 2002—this is only 1.9% of the projected on-line volume 
of 221,100 renewals. Given that there are about 7 million vehicle licences renewed 
every year, the total Internet penetration for this service was approximately 0.05%. 


¢ Although the vehicle licence renewal service was designed to be “soft-launched”—that 
is, with minimal publicity for the first two months of operation to ensure the system 
could handle the expected transaction volumes—the communications strategy 
recommended that higher-profile promotions were to follow the initial two-month 
period. However, we saw little evidence of public promotion of this service. 


At the Ministry of Consumer and Business Services, we noted that a draft 
communications plan had been prepared for the Ontario Business Connects program 
(the program allows business registration and other related services to be conducted by 
electronic means). The plan contained a detailed promotion strategy, but no target dates 
had been established for any of the planned activities. It is therefore unclear how many, if 
any, of these planned components will be executed in time to support the government's 


2003 ESD targets. 


In addition, at the same ministry, we noted that a five-year contract with a private-sector 
consortium was in place to improve public access to 24 government electronic services. 
We reviewed the procurement and evaluation processes the Ministry followed to ensure a 
fair, open, and competitive process in the awarding of this consortium contract. We 
concluded that government directives were complied with and good business practices 
were followed. However, under the contract, the consortium is responsible for developing 
appropriate marketing and communication plans related to these services. The first 
promotion plan prepared by the consortium, in our view, provided insufficient detail as 
to how public awareness and use of these services was to be generated, even though 
performance targets set under this contract are very aggressive and suggest that a 
corresponding aggressive promotion campaign will be necessary. The contract covers 
some 11.7 million annual transactions with the public, and it is estimated that, by the 
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end of the five-year term, a significant portion of these transactions will be provided through 
the electronic means developed by the consortium—-specifically, between 5.3 million and 9 
million transactions (45% to 77% of total transactions). These transactions include the on- 
line vehicle licence renewal service discussed earlier that has essentially achieved no Internet 
penetration. 


The Ministry of Training, Colleges and Universities has had considerably more success in 
promoting its on-line student assistance program. Student loan summary statistics over 
the last four-year period are presented in the following table. 


Ontario On-line Student Loan Applications Statistics, 
1998/99-—2001/02 


: School Year : 
1998/99 1999/00 — 2000/01 


Total loan applications 241,124 red fo [a eNi, foo 195,087 





On-line loan applications 45,538 66,361 915275 110,722 


Percentage of applications 
submitted on-line 





19% 29% 43% 57% 





Source of data: Ministry of Training, Colleges and Universities 


Growth in the on-line loan program has been quite impressive with over 50% of the 
student population applying for assistance on-line in 2001/02. There are a number of 
possible explanations for this, including the likelihood that students in general have 
higher computer literacy rates than the general population as well as the efforts the 
Ministry has exerted to communicate the availability of the on-line service and to 
integrate this activity into its overall program delivery. Another factor that contributed to 
the success of the on-line application was the fee structure applied to this initiative. While 
a $10 processing fee is required with the submission of a loan application made via 
traditional channels, this fee is waived for on-line applications. 


We believe this type of differential fee structure may be a useful approach to promoting 
Internet activities in other government areas, particularly where the Internet provides a 
lower, long-term cost for the delivery of government services. We noted that an ESD 
pricing strategy that MBS recently developed would appear to allow for this type of 
promotion where appropriate (MBS’s pricing strategy outlines guidelines for appropriate 
fees on government services delivered electronically). 


We also noted that such pricing approaches were supported by a consulting firm hired by 
the Ontario government in 2000 to conduct a review of ESD in other jurisdictions. 
Specifically, the consulting firm’s report made reference to a case study in Arizona where 
the majority of user complaints about motor vehicle registration services related to a $6.95 
transaction fee, noting that: “In less than a year, Arizona eliminated the fee and witnessed 
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electronic registration renewals increase by 21%.” The consulting report also cited other 
sectors, such as financial services, that have long used differential pricing to increase usage of 


FSD: 








Recommendation 


To maximize the public’s use of electronic service delivery (ESD), Management 
Board Secretariat should: 


¢ develop and deliver an ongoing communication campaign that builds 
consumer awareness of ESD and promotes its use; 

¢ work with ESD ministries to help them ensure consistent messaging and 
co-ordination of promotional efforts; 

¢ where specific penetration targets are set for particular ESD applications, 
help ministries develop commensurate promotional strategies to achieve 
those targets; and 

¢ consider differential pricing strategies where ESD offers a promise of 
providing significant long-term cost savings in program delivery. 








Management Board Response 


Management Board Secretariat (MBS) agrees that the take-up of our 
electronic services will not happen without active communication and 
marketing and is refining its corporate communications strategy to 
emphasize marketing of government electronic channels and services. 


Significant work is underway at MBS and the Ministry of Consumer and 
Business Services to implement approved communications plans for 
e-government and integrated service delivery. Where a particular ESD 
application with a large prospective customer base is being launched, the 
ministries are encouraged to actively market their e-services to prospective 
clients, ensuring that corporate messaging and consistent practices are 
used. 


Work is also underway to explore the feasibility of various policies aimed at 
encouraging take-up of electronically delivered services. As the Provincial 
Auditor has noted, MBS is implementing an approved channel pricing 
policy, and several pilot projects are to be undertaken in this fiscal year to 
help evaluate these models and identify scenarios where they will have the 
best impact. 
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DEVELOPMENT AND DELIVERY 
OF ELECTRONIC SERVICES 


As mentioned earlier, the October 2000 ministry action plans identified high-impact 
services for each ministry. At the ministries we visited, we focussed on the Web-based, 
front-end systems that assisted the ministries in providing ESD services to the general 
public. Specifically, we reviewed the project planning, execution, and monitoring 
processes for three of these service improvement projects as well as for one of the 14 
projects for which Cabinet approved additional funding. We noted that, for these high- 
priority projects, while the ministries had implemented a number of sound project 
management practices, with respect to security practices and service availability, there was 
some room for improvement. 


Security 


In reviewing security issues relating to ESD initiatives, we examined both the security of the 
government network and the security of systems for the areas we reviewed at the ministries 
we visited. The government network is the telecommunications infrastructure that provides 
connections within and between ministries, as well as between the government and the 
public. Ministry systems comprise various application programs and databases. 


NETWORK SECURITY 


In February 2001, MBS entered into an agreement with a private-sector company to 
provide for the management of the government’s network. The purpose of this project is 
to develop a robust, high-capacity, and cohesive telecommunications network that 
supports effective internal communications and links to the broader public sector. The 
project consists of many components, and the process of transferring responsibility for 
many of these components was in transition at the time of our audit. MBS expected this 
project to be completed by the end of summer 2002. 


We found that a full security audit of this project could not be conducted at this time 
because significant steps in the process were still outstanding. Accordingly, we limited our 
review to the security provisions in the contract with the private-sector company. Since 
these security provisions were linked to MBS’s new security guidelines and standards for 
computer environments, we reviewed these guidelines and standards, which were recently 
developed as an extension of the Management Board Information and Information 
Technology Security Directives. We also evaluated the administrative arrangements under 
which the contract will be managed, including the roles and responsibilities of key 
stakeholders, such as the company (and its sub-vendors), the MBS corporate security 
group, and other monitoring and governance committees, 


The provisions in the contract and the government security guidelines are sufficient to 
provide reasonable assurance that the government network will be properly protected. 
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However, MBS will need to obtain assurance—through ongoing monitoring and audits of 
third-party service providers—that these guidelines have been effectively implemented. The 
contractual arrangements contain stipulations enabling such monitoring and regular 
independent audits, and MBS intends to commission its first audit later this year. 


The same private-sector provider also manages and monitors network security for the 
government Internet gateway. This gateway is the main entry point for users accessing all 
government electronic services. MBS maintains an intrusion detection service on this 
gateway to detect any unauthorized and/or inappropriate access attempts or attacks. 
During our audit, we noted that live monitoring only took place during office hours. 
Therefore, security threats that occur in the evenings or on weekends may not be 


promptly identified and dealt with. 


As part of our review, we also looked at the standard security requirements for other 
government jurisdictions offering on-line services to the public and noted that Web-based 
general security standards to process transactions was similar. However, some 
international jurisdictions have incorporated higher levels of security for certain 
designated high-risk or confidential transactions by requiring that users provide a 
previously issued personal digital certificate—an encrypted form of electronic 
identification that authenticates a user’s identity—before allowing them to complete 
certain transactions. 


SYSTEM SECURITY 


In February 2001, Management Board of Cabinet directed all ministries to complete a 
formal threat risk assessment (TRA) for all “high-trust” ESD initiatives. A TRA helps 
ministries identify potential high-risk areas (where the nature of the system or the data 
necessitates a greater degree of protection—for instance, for reasons of confidentiality). 
This, in turn, allows ministries to develop controls to mitigate such risks. MBS has 
produced a guide to assist ministries in conducting these assessments. We noted that the 
Ministry of Transportation and the Ministry of Training, Colleges and Universities had 
not completed TRAs for the electronic services we reviewed. Therefore, there was no 
assurance that these ministries had appropriate levels of protection applied to all sensitive 
information. 


We assessed computer security arrangements for the ESD systems we reviewed at the 
ministries we visited. These systems included the Web-based applications and hardware 
required to connect the ministries’ internal applications and database systems with the 
Internet. We concluded that there were several areas where security could be improved: 


°  Credit-card information received from customers of the Ministry of Consumer and 
Business Services is transmitted internally in clear text using a local-area network. 
While these transmissions take place behind a government firewall and industry- 
accepted safeguards are built into the transmission process to external parties, additional 
steps can be taken to enhance security in this area. 
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The Ministry of Transportation uses GONET, the government of Ontario’s internal 
wide-area network, which is behind a government firewall, to transmit credit-card and 
driver's licence information in clear text, not encrypted. We also noted that security 
measures that enhance data confidentiality—such as a public key infrastructure (PKI) 
system—were not in place for these transmissions. 


At the ministries of Consumer and Business Services and Training, Colleges, and 
Universities, duties and responsibilities in the administration and operation of systems 
were not adequately segregated. Some individuals were assigned multiple job 
functions that granted incompatible system rights, thereby increasing the risk of data 
manipulation. 


These two ministries also did not have adequate controls to protect user accounts and 
system resources. For instance, the Ontario Business Connects system did not revoke 
user accounts after a number of unsuccessful log-in attempts. As well, file protection 
and user password management for both ministries did not comply with government 
standards to ensure appropriate levels of security were maintained. For example, the 
expiration limits for user passwords exceeded standards at both ministries. 


The Ministry of Training, Colleges and Universities did not have formal security 
administration procedures and tools to ensure active security monitoring. As well, at 
both the Ministry of Training, Colleges and Universities and the Ministry of 
Consumer and Business Services, inactive accounts were not being deleted from 
systems promptly. In one program area at the Ministry of Consumer and Business 
Services, we found 180 inactive user accounts. We were informed that these accounts 
could not be deleted while the legacy system architecture remains in place without 
compromising data integrity. 


Security over the computer room at Ontario Business Connects and over the 
electrical room containing routers at the Ontario Student Assistance Program could 
be improved. Unauthorized individuals could potentially enter these facilities and 
access sensitive information. 








Recommendation 


To ensure that confidential data is better protected against unauthorized 
access and potential tampering, Management Board Secretariat and the 
Ministries should: 


| 


centrally establish an intrusion detection service providing coverage 24 
hours a day, seven days a week, to ensure continuous monitoring of the 
Ontario government network; 


explore the possibility of using more secure mechanisms, such as personal 
digital certificates, to authenticate the identity of individuals transacting 
with the government through the Internet; _ 
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¢ consider completing threat risk assessments for all major existing services 
delivered electronically to ensure data is adequately protected; 


e consider cryptography or other controls to secure data transmitted over 
the government’s internal and external networks until alternative 
arrangements, such as a centrally administered public key 
infrastructure system, are in place to ensure data confidentiality and 
integrity; 

¢ segregate system duties such that individuals are not assigned 
incompatible system rights; and 


¢ implement more rigorous controls over system passwords and user 
accounts to protect system resources and user accounts. 


Management Board and Ministries Response 


Management Board Secretariat (MBS) has received approval from 
Management Board of Cabinet to implement round-the-clock monitoring of 
the Government of Ontario OPS [Ontario public service] private network, 
365 days a year. Additional staff are currently being recruited to support 
the continuous monitoring of OPS information and infrastructure assets, 
and funding has been allocated to expand the intrusion detection network, 
providing enhanced detection and response capabilities. 


The Information Protection Centre is expanding its partnership with other 
accredited early warning agencies, forming a network to gather advanced 
intelligence information on emerging electronic threats. 


Ontario has one of the largest public key infrastructure (PKI) deployments 
in North America and is positioned to use this security technology to 
provide enhanced electronic authentication. MBS will continue to provide 
advice to clusters and ministries and to assist them in leveraging the 
current PKI infrastructure and other new security practices and tools, such 
as cryptographic controls that are available to them. 


MBS routinely provides threat risk assessment (TRA) advice on any new 
system under development in ministries and welcomes the 
recommendation to undertake security reviews of existing systems: 


e The Management Board of Cabinet has approved a plan for MBS to 
participate in the development of TRAs for all mission-critical systems. 

e MBS is currently working with I&IT clusters and with ministries to 
prioritize the systems that will be reviewed. 


The Ministry of Training, Colleges and Universities completed its threat risk 
assessments in August 2002. The Ministry of Transportation will complete a 
formal threat risk assessment following MBS guidelines for all future 
projects. 


The Ministry of Consumer and Business Services places a high priority on 
securing its electronic channels and services. For instance, where government 
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systems interface with financial institutions, the Ministry uses the security 
protocols adopted by the international financial institution community, and 
credit-card information received from customers is transmitted in accordance 
with approved security controls and practices internally. The Ministry of 
Transportation uses encryption internally, behind the firewall, between the 
client and the Web server using industry-standard encryption methodology 
and is investigating the cost of encrypting data end-to-end. 


The ministries of Consumer and Business Services and of Training, Colleges 
and Universities have already taken action to address some of the Provincial 
Auditor’s concerns. 


MBS will ensure ministries follow proper security management practices and 
will advise clusters and ministries on the importance of the segregation of 
duties to limit security breeches. Annually, ministries undertake a data security 
self-assessment. MBS will advise I&IT clusters and ministries of required 
security controls and policies and will follow up to confirm that these are being 
implemented properly by I&IT clusters and ministries. 








service Availability 


To be viable and credible, ESD services should be designed to operate on a high availability 
basis. Service disruptions should be minimal. We reviewed the data backup and disaster 
recovery processes of the Web-based applications at our selected ministries and had the 
following concerns: 


* At the Ministry of Transportation, the same computer server was used for over 15 on- 
line applications, even though generally, servers support approximately one to two 
moderately sized applications. The overloading of this server has contributed to 
system downtime and performance issues over the last year. At the time of our audit, 
the Ministry was preparing to add an application for Internet road-test booking onto 
this same server. This can only further aggravate service availability. In addition, the 
Ministry did not have procedures, system utilities, and diagnostic tools in place to 
monitor system performance in real time. Since there were performance capacity 
issues with the existing system, such monitoring would help prevent sudden system 
outages. 


¢ The Ministry of ‘Training, Colleges and Universities stores all new and amended 
application data for the Ontario Student Assistant Program on a Web server and 
transfers this data to a back-end application system twice daily. A number of 
safeguards are in place to protect this data between transfers. However, despite these 
safeguards, there remains the possibility of loss of student loan application data if the 








Web system fails between scheduled file transfers. Real-time backup could eliminate this 
risk. | 
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Recommendation 


To ensure a high availability of electronic services and that all collected client 
data remains complete and accurate: 


¢ Management Board Secretariat should develop standards and policies to 
address systems availability; 


e the Ministry of Transportation should review its hardware performance and 
capacity needs to ensure its systems can provide appropriate service levels 
to the public; and 


e the Ministry of Training, Colleges and Universities should consider 
instituting a process of real-time backup for the application data 
relating to the Ontario Student Assistance Program. 


Management Board and Ministries Response 


Management Board Secretariat (MBS) will ensure that, where appropriate, 
sufficient redundancy is built into the infrastructure to support high 
availability for those mission-critical applications that need to be available 
around the clock, such as systems that support police and ambulance 
services. MBS will develop policies and standards to ensure new systems 
are architected and designed to enable high availability, where business 
requirements dictate, and will work with ministries to upgrade existing 
systems. 


The Ministry of Transportation is rigorously replacing obsolete and 
overloaded servers. The server referred to is included in this replacement 
activity. The target to refresh the overloaded application and Web servers is 
the fourth quarter of 2002/03. Following the introduction of the replacement 
server, the Ministry plans to improve its proactive monitoring using modern 
diagnostic tools and system utilities. 





The Ministry of Training, Colleges and Universities feels the risk of data 
loss is extremely low with its current system. Data collected through the 
Web-server system is protected by a disk protection system that ensures 
steady, ongoing processing of information in the case of a drive failure. In 
addition, essential data collected from students on their applications is 
archived between the twice-daily backups to allow full recovery of data in 
the case of a catastrophic loss of application data. 
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MANAGEMENT BOARD SECRETARIAT 

AND THE MINISTRIES OF THE ENVIRONMENT, 
FINANCE, HEALTH AND LONG-TERM CARE, 
NATURAL RESOURCES, AND 

PUBLIC SAFETY AND SECURITY 


3.06—Consulting Services 





BACKGROUND 


Consulting services, as defined under the Management Board of Cabinet Directive on 
Consulting Services (Directive), are services provided for a fee, on the basis of a defined 
assignment, and relating to management consulting, information technology (IT) 
consulting, technical consulting, and research and development. 


Over the past five years, there has been a substantial increase in annual consulting services 
expenditures at Ontario ministries, from $271 million in 1998 to $662 million in 2002. 
The following graph illustrates this increase. 


Annual Consulting Services Expenditures at Ontario Ministries, 1998-2002 
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Source of data: Ministry of Finance 
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The increase has been particularly significant for IT and management consulting services. 
Expenditures for IT consulting services have tripled from $100 million to $313 million 
during the past five years, while those for management consulting have increased almost 


four-fold, from $40 million to $152 million. 


With respect to the planning, acquisition, and management of consulting services within the 
Ontario government, Management Board Secretariat (MBS) is primarily responsible for: 


¢ developing and evaluating the effectiveness of the Directive and recommending any 
changes to the Management Board of Cabinet; 


* assisting ministries in the development of their administrative procedures in support of 
the Directive; and 


¢ reviewing ministry submissions and reports to evaluate ministry performance with 
respect to compliance with Directive requirements. 


The Directive outlines four main principles, which are summarized as follows: 


° Competition: Consulting services shall be acquired competitively, through open and 
transparent procurement procedures, to meet specified needs with the objective of 


obtaining the best value for the funds to be expended. 


¢ Access: Access for suppliers to compete for ministry consulting assignments is to be open, 
fair, and consistent. When acquiring consultants, ministries must avoid any conflict of 
interest, must not permit a supplier to gain a monopoly for a particular kind of work, 
and must not continuously rely on a particular outside organization. 


¢ Fair and equal treatment of suppliers: Suppliers of consulting services shall be treated in a 
fair, equitable, and responsible manner. 


¢ — Responsible management. Ministries shall co-ordinate, monitor, and control the 
combined efforts of internal and external resources to ensure satisfactory completion of 
consulting assignments on schedule and within budget. When appropriate, a transfer of 
knowledge must occur from consultant to staff to avoid a continuous reliance on 
consultants. 


These principles are in turn reflected in the following mandatory requirements: 


° Assignment definition, justification, and approval: Before seeking assistance from 
suppliers of consulting services, the ministry must ensure that the assignment is well 
defined and justified and that appropriate approvals are obtained. 


° Competitive acquisition: The estimated ceiling price of an agreement (including fees and 
expenses) determines the prescribed acquisition procedure. Specifically: 


- Ifthe estimated ceiling price is less than $25,000, the ministry determines the 
appropriate procedure. However, competition must not be avoided by awarding 
the same consultant continuous agreements, each less than $25,000 but totalling 
more than $25,000, unless each assignment is unique and different and the terms 
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of reference of each new assignment change substantially from the previous 
assignment. 


- Ifthe estimated ceiling price is $25,000 or more, all consulting services must be 
acquired through a competitive process that must include at least a written request 
for proposals; a written evaluation of proposals; a written agreement; and, upon the 
request of bidders, bidder debriefing. Ministries must select from among qualified 
bidders the consultant with the lowest evaluated cost. 


- Ifthe estimated ceiling price is $100,000 or more, an open call for tenders must be 
issued. 


¢ Approval of supplier selection: When the ceiling price for an assignment (including any 
follow-on assignments) is $100,000 or more, approval in writing of the selected 
supplier, the selection process, and the proposed agreement must be obtained from the 
Deputy Minister or designate. When the estimated price is $500,000 or more, the 
approval of both the Deputy Minister and Minister must be obtained. 


¢ Agreements: The ministry and the supplier must formally define the responsibilities of 
both parties with respect to the assignment through a written agreement. Further, the 
ceiling price of an agreement must not be exceeded if the terms and conditions of the 
agreement remain unchanged. 


In all cases in which changes to the terms and conditions for any agreement increase the 
ceiling price, the following need to both be documented and receive the prior approval 
of the Deputy Minister or designate: the changes and/or additions to the agreement; 
the method used to arrive at the revised ceiling price; and the reason why the need for 
changes and/or additions were not foreseen prior to agreement signature. The prior 
approval of both the Deputy Minister and Minister is required for any changes to the 
ceiling price that cause the total value of the agreement to reach or exceed $500,000. 


¢ — Assignment documentation: Ministries must retain appropriate documentation of each 
assignment for reporting to MBS and for future reference. Ministries must be prepared 
to provide a report to MBS on their ability to effectively manage the process of 
planning, acquiring, and managing consulting services. 


In addition to the requirements set out in the Directive, MBS has established corporate 
vendor-of-record (VOR) arrangements with multiple vendors. These are contractual 
arrangements for the ongoing acquisition of commonly purchased goods and services, 
including consulting services, over a specified term that are available to all ministries. 


AUDIT OBJECTIVE AND SCOPE 


The objective of our audit of consulting services was to assess whether six selected ministries 
had adequate systems and procedures in place to ensure that consulting services were 
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acquired and managed with due regard for value for money and in compliance with 
corporate policies. 


The scope of our audit included the following six ministries (the Ministries): Management 
Board Secretariat and the ministries of the Environment, Finance, Health and Long-Term 
Care, Natural Resources, and Public Safety and Security (Public Safety and Policing Services 
Divisions only, constituting the former Ministry of the Solicitor General). Our audit work 
included interviews with appropriate staff at the Ministries, a review of the Directive and 
other relevant administrative procedures, and an examination of consulting services 
contracts, invoices, and other related documents. 


Prior to the commencement of our audit, we identified the audit criteria that would be used 
to conclude on our audit objectives. These were reviewed and agreed to by senior 
management at the Ministries. 


Our audit was performed in accordance with the standards for assurance engagements, 
encompassing value for money and compliance, established by the Canadian Institute of 
Chartered Accountants, and accordingly included such tests and other procedures as we 
considered necessary in the circumstances. 


During the course of our audit, we worked with the Internal Audit Services for the Ministry 
of Finance and were able to rely on their work to reduce the extent of our work at the 


Ministry. Our audit field work was substantially completed by March 2002. 


OVERALL AUDIT CONCLUSIONS 


Our audit identified numerous instances where the Ministries did not comply with directive 
requirements and MBS guidelines. As a result, we concluded that, in many respects, 
consulting services were not acquired and managed with due regard for value for money. 
The following is a summary of our major concerns. 


Use of Consultants at Ministries 


Heavy dependence on the use of consultants: Hundreds of consultants were engaged at per 
diem rates that were on average two to three times higher than the salaries of ministry 
employees performing similar duties. For instance: 


¢ At the Ministry of Public Safety and Security, over half of the IT workforce was made 
up of consultants. In addition, more than 40 of these consultants were former ministry 
employees who had left the Ministry in 2001 and returned within a few days at per 
diem rates that were more than double their salaries as employees. 


Awarding continuous agreements with little or no change to the original deliverables: Ministries 
often entered into contracts that started with the consultant providing its expertise for a 
short period of time, usually on a three- to six-month contract, after which the contracts 
were extended with little or no change to the original deliverables. For example: 
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¢ A consultant was initially engaged by the Ministry of the Environment to provide 
technical consulting services for three months at a rate of approximately $540 per day, 
with total remuneration not to exceed $25,000. Over the following three years, the 
consultant’s contract was renewed or extended seven times without any change in the 
contracted deliverables, for total remuneration of $455,000. 


¢ A consultant engaged by the Ministry of Health and Long-Term Care on a six-month 
assignment at $96,000 was awarded successive contracts that extended the term to two 
years, resulting in a total price of $360,000. The consultant was eventually replaced 
with a full-time staff member at an annual salary of approximately $60,000. 


Lack of knowledge and experience for carrying out senior-level responsibilities within ministries: 
Some ministries had to engage senior-level consultants at high per diem rates to oversee 
projects and manage the work of other consultants. For example: 


¢ As at March 31, 2002, the Ministry of Health and Long-Term Care had engaged 
about 15 senior consultants who charged over $1,200 per day on average in a 
management capacity, compared to approximately $340 per day for senior ministry 
employees at the equivalent level. 


Lack of competition in acquiring IT consultants: \n the development and implementation of 
multi-million-dollar IT projects, the Ministries often engaged consultants on a per diem 
basis to do the work instead of calling for open tender. This practice did not ensure that the 
most qualified consultants were acquired at the best available price and that all suppliers of 
consulting services were given fair access and treated in an open and transparent manner. 
Examples of our findings are provided in the following table. 


Information Technology Projects for Which an Open Tender Process Was Not Followed 


. nd Re) (21015 Of e.-] bo) 
Ministry Project (estimated or actual) Other Concerns 


Environment Environet (three of $1.1 million—$4.2 million 
six components) (estimated) 


Natural Resources MNR ON-Line $1.9 million (actual) a no estimate made of cost and 
March 31, ie time required to complete the 
project 
Health and Long-Term Smart System for $12.7 million (actual) at 
Care Health March 31, 2002 


Integrated Services | $8.5 million (actual) at original estimate was $6 million; 
for Children March 31, 2002 estimated additional $3 million 
Information System required to complete the project 











a project to define $5.9 million (estimated) 
and implement 

quality-assurance 

processes 


Public Safety and Security | IT Service $3 million (actual) at consultant sub-contracted some 
Management March 31, 2002 work to another consultant on 
Project current VOR list 


Management Board a management $1.4 million (estimated) | project cancelled by MBS after 
Secretariat improvement $467,000 was spent 
project : 





Sources of data: Management Board Secretariat and Ministries of the Environment, 
Health and Long-Term Care, Natural Resources, and Public Safety and Security 
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Risk assumed by engaging consultants on a per diem basis: By compensating consultants on a 
per diem basis and not on the basis of a fixed price and fixed deliverables, the Ministries 
assumed the risk of consultants not delivering their work on time and would often be forced 
to pay the costs of missed deadlines and cost overruns, even when these problems may have 
been caused by unsatisfactory performance and inefficiencies on the part of the consultants. 


Lack of competition in acquiring other consultants: In the acquisition of non-IT consultants, 
we found problems such as the following: 


¢ At MBS, a $1.7 million contract to provide management consulting services relating to 
the government's data network was awarded to a firm without any competition. An 
open tender process would have been prudent and appropriate in view of the large 
amount paid to this firm. 


Weaknesses in the competitive selection process: We found instances of weaknesses in the 
competitive selection process, such as: 


¢ At the Ontario SuperBuild Corporation—an agency of the Ministry of Finance—a 
calculation error resulted in a consultant being awarded a contract as a financial advisor 
for $681,000. The consultant may not have received the contract if the error had not 
been made. 


¢ In another case, SuperBuild Corporation’s poorly defined evaluation criteria led to the 
awarding of a $3-million consulting contract, which was double the price bid by the 
lowest bidder. 


An agreement not reflecting the scope of consulting work: A forensic accounting firm was 
engaged by MBS to review certain real-estate transactions entered into by the Ontario 
Realty Corporation (ORC). Its estimated fees ranged from $150,000 to $500,000. 
However, the firm was paid almost $6 million out of ORC’s budget for expanded work, 
and a new contract was not entered into to reflect the revised scope and objectives of the 
ongoing investigative work. 


Not ensuring that consultants’ provincial taxes were in good standing: The Ministries frequently 
did not ensure that consultants’ provincial taxes were in good standing as required by their 
own policies and by statements made in the 1996 Ontario Budget Speech. For example: 


¢ At MBS, two consultants who received contracts were not registered with the Ministry 
of Finance, and two other consultants had tax arrears of approximately $110,000 and 
$35,000 respectively. 


¢  Atthe Ministry of Public Safety and Security, two consultants who received contracts 
were in default for not filing corporate tax returns. 


Significant weaknesses in controls over payments to consultants: Instances of weak controls 


included: 
¢ At the Ministry of the Environment and the Ministry of Public Safety and Security, 


numerous payments were made to consultants that exceeded the ceiling price of 
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contracts, and there was no evidence of prior approvals by the Deputy Minister or 
designate as required by the Directive. 


¢ At the Ministry of Public Safety and Security, a consulting firm was engaged to monitor 
and operate computer systems onsite and at remote locations around the province. The 
arrangement continued for seven months after the contract expired, and the firm was 
paid, in total, $200,000 above the contract amount. 


¢ At MBS, one consulting firm was reimbursed for meal charges at ten times the rates 
allowed to government employees, another consultant was mistakenly paid twice an 
amount over $14,000, and no supporting documents were found for over $36,000 in 
charges paid to another consultant. 


¢ MBS permitted one consultant’s rates to increase significantly, from $725/day in 
April 2000 to $1,800/day in May 2000 and to $2,600/day in September 2000 
without documented rationale for these large increases in the consultant’s rate. In 
another case, a consulting firm, which received almost $450,000 for an engagement, 
charged the Ministry of Natural Resources hourly rates of $393 and $218 per hour, 
even though the rates quoted for the engagement were $300 and $200 per hour 
respectively. 


Direction To Ministries 


The Directive provides adequate direction and guidance to ministries to ensure that, if 
followed, the planning, competitive acquisition, and management of the use of consulting 
services are undertaken with due regard for value for money. However, we identified a 
number of areas where the VOR arrangements for consulting services established by MBS 
were inconsistent with the Directive and were insufficient to ensure that ministries using 
vendors of record receive value for money. This was of particular concern because on 

May 1, 2000, all ministries were informed by MBS that, effective immediately, the use of 
the corporate VOR arrangements would be mandatory for the acquisition of all consulting 
services. The following is a summary of our major concerns. 


Approval not obtained for changing directive requirements: MBS established the threshold 
amount of $500,000 for certain VOR arrangements before an open tender is required. 
This threshold amount is five times higher than the threshold amount of $100,000 in the 
Directive. In other cases, MBS did not establish any threshold amounts whatsoever. In 
establishing a higher amount, it had not obtained approval from the Management Board of 
Cabinet and also had not recommended changes to the Directive to reflect the higher VOR 
competitive tendering thresholds. 


Lack of equal opportunity for consultants to bid on assignments: The VOR lists include 
consultants in the same job categories that charge significantly different rates—rates can 
vary by as much as $2,000 per day. However, MBS guidelines did not require that 
ministries give qualified consultants on a VOR list an equal opportunity to demonstrate that 
they can satisfactorily complete a given assignment at the lowest cost. 
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Unclear guidelines on requirements for open tender: The practice of some ministries was to 
avoid open tender by subdividing multi-million-dollar IT projects into smaller individual 
assignments. For example, at the Ministry of Health and Long-Term Care, an IT project 
that had cost $10.6 million to date had about 50 consultants working on smaller 
assignments of less than $500,000 each. The Ministry advised us that this practice was in 
compliance with VOR requirements. There is a need to clarify the VOR guidelines to 
ensure that the procurement process is open and transparent and that the most qualified 
consultants are obtained at the lowest costs using an open tender process whenever possible. 


OTHER MATTER 


We had concerns with respect to the engagement of creative communications consultants at 
the Ontario Realty Corporation (ORC). For example, one firm had billed ORC over 
$1 million, but letters of agreement existed only for billings of $210,000. 


DETAILED AUDIT OBSERVATIONS 
DIRECTION TO MINISTRIES 


The Management Board of Cabinet Directive on Consulting Services (Directive) sets out 
corporate policies and the requirements that ministries must follow in planning for, 
competitively acquiring, and managing consulting services with due regard for value for 
money. Ministries must obtain prior approval from the Management Board of Cabinet for 
any exemption from the requirements contained in the Directive. 


Further direction to ministries with regard to acquiring consulting services is provided 
through the contractual VOR arrangements with multiple vendors established by MBS. 
These VOR arrangements are for the ongoing acquisition of commonly purchased goods 
and services, including consulting services, over a specified term and are available to all 
ministries. MBS’s Procurement Policy and IT Procurement Branch establishes and manages 
corporate VOR arrangements for information technology (IT) consultants, and the Shared 
Services Bureau establishes and manages corporate VOR arrangements for non-IT 
consultants. 


On May 1, 2000, all ministries were informed by MBS that, effective immediately, the use 
of the corporate VOR arrangements established by MBS would be mandatory for the 


acquisition of all consulting services. 


Establishing Vendor-of-record Arrangements 


To establish VOR arrangements with various categories of consultants, MBS begins with an 
advertised request for proposals for various types of consulting services used by the 
government, and each response from an interested consultant is reviewed based on the 
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consultant’s experience, qualifications, and the billing rates the consultant submits for a 
specific type of work. Approved VOR lists are maintained for each specific type of 
consulting service and include all consultants who successfully met the criteria for inclusion 
on the lists. Typically, there are many consultants for each type of service. The VOR lists help 
ministries achieve administrative efficiencies in that much of the work of identifying 
prospective consultants and evaluating their credentials has already taken place. 


Ministries are required to select from the appropriate list the consultant that best meets the 
needs of the assignment at the lowest cost. For large assignments, the ministry is required to 
receive competing proposals from several consultants, with the selection being based on the 
merits of each proposal. The cost of the assignment can vary significantly between 
consultants, depending on the consultant’s approach, the consultant’s estimates of the 
amount of time and work required to complete the assignment, and the consultant’s per 


diem rates on the VOR list. 


We noted several areas where the VOR arrangements require improvement to ensure that 
they reflect the requirements for fairness, openness, and competitiveness in the Directive 
that help ensure ministries achieve value for money. 


¢ The Directive states that if the estimated ceiling price of an assignment (including fees 
and expenses) is $25,000 or more, all consulting services must be acquired competitively 
through a written request for proposals from potential suppliers. Ideally, the acquisition 
procedure will result in consideration of three or four valid proposals. If the estimated 
ceiling price is $100,000 or more, a call for open tender is required. However, we noted 
that, for several categories of consultants, the threshold amount set by MBS for the use 
of VOR lists was often significantly higher than the amount specified in the Directive. 
For example, certain VOR arrangements allow a ministry to select a consultant of its 
own choosing without any competition unless the ceiling price of an assignment is more 
than $100,000, contrary to the Directive’s threshold amount of $25,000. Similarly, the 
VOR arrangement for IT consultants allows ministries to select from among the 
consultants on the list without calling for open tender unless the ceiling price of an 
assignment is more than $500,000, contrary to the Directive’s threshold amount of 
$100,000. The VOR arrangement for alternative-service-delivery consulting services 
did not even have a threshold amount beyond which competitive selection of 
consultants and calling for open tender is required. 


In all these cases where the threshold amounts beyond which competitive consultant 
selection is required were higher under VOR arrangements than in the Directive, there 
was no documented rationale to justify the higher threshold amounts. In addition, MBS 
had neither sought approval from the Management Board of Cabinet to set the higher 
threshold amounts nor had it recommended changes to the Directive to reflect the 
higher tendering requirements when the VOR process is involved. 
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* The VOR lists for each specific type of service include a large number of consultants 
who charge significantly different rates. For example, the following chart compares the 
lowest and highest rates that vendors on the same VOR lists ask for their services. 


Ranges of Per Diem Rates on Two VOR Lists 


Lowest Highest — , 
Per ‘Per Difference 
Diem  #ODiem i$) 
Rate ($) Rate ($) 8 
VOR List for IT Services—Oracle Environment (Greater 
Toronto Area) ee 


Job Category 


Project manager/leader 435 2,340 1,905 
Technology architect 435 1,885 1,450 
Applications architect 400 1,885 1,485 
Data architect/modeler 375 1,600 1275 
Systems analyst 326 1,415 1,089 
Programmer/analyst 350 1,290 940 
Database administrator 218 15150 932 
‘Services — —. 2 ee 
Partner 1,200 3,350 2ADO 
Principal 1,200 2,883 1,683 
Project manager 900 2,240 1,340 
Senior consultant 800 1,790 990 
Consultant 700 15595 895 
Business analyst 471 1,268 797 


e Service Delivery Consulting 





Source of data: Management Board Secretariat 


The VOR list for alternative-service-delivery consulting services has 22 firms whose per 
diem rates vary by over $2,000, while the VOR list for IT services—Oracle 
environment (Greater Toronto Area) has 116 firms, and the per diem rates vary by 
almost $2,000. Given such large numbers of firms and varying rates, it is particularly 
critical that a fair and open competitive selection process be followed by ministries to 
ensure that the consultant who meets the needs of an assignment will be acquired at the 
lowest cost. However, MBS guidelines to ministries did not require that qualified 
consultants on the VOR lists be given an equal opportunity to submit a proposal to 
demonstrate that they can satisfactorily complete the assignment at the lowest cost. We 
noted instances where ministries requested proposals from only one or a few consultants, 
who charged rates at the higher end of the per diem range, and excluded from 
consideration, perhaps for reasons of unfamiliarity, other consultants who may be 
qualified and offered a much lower rate for their services. 
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For some ministries, we noted that several multi-million-dollar projects, particularly IT 
projects, were subdivided into individual smaller assignments of less than $500,000. 
These ministries did not call for open tender for the assignments but rather selected 
consultants from the VOR lists on a per diem basis to complete the assignments. In that 
regard, we noted that guidelines provided to ministries by MBS were unclear with 
respect to when open tender is required. Our two concerns were that guidelines do not 
address the need for ministries to better define large projects that should be acquired 
using open tender, and they do not address the need for ministries to require fixed 
deliverables at a firm price in order to ensure that consultants assume responsibility and 
some of the risk relating to their work. 


With respect to the first concern, in the view of some ministries, the VOR requirement 
that consultants for IT assignments of $500,000 or more be acquired through open 
tender does not mean that ministries cannot subdivide multi-million-dollar projects into 
many individual smaller assignments; as long as each assignment is distinct and has an 
estimated ceiling price of less than $500,000, the ministries can select consultants from 
the VOR lists for the assignments and not follow an open tender process. For example, 
at one ministry, we noted that about 50 consultants were working on an IT project that 
had cost $10.6 million to date. The ministry indicated to us that each of the consultants 
was working on an individual assignment of less than $500,000 and that therefore the 
ministry was following the VOR requirements in not requesting open tender. There is a 
need to clarify the VOR guidelines to ensure that the procurement process is open and 
transparent and that the most qualified consultants are obtained at the lowest costs using 
an open tender process whenever possible. 


With respect to the second concern, because consultants were engaged on a per diem 

basis and were not responsible for fixed deliverables, the ministries assumed the risk of 

consultants not delivering their work on time and were often forced to pay the costs of 
missed deadlines and cost overruns, even when these problems may have been caused 

by unsatisfactory performance and inefficiencies on the part of the consultant. 


Ministries are not required to report to MBS on the extent to which VOR 
arrangements are used. In addition, only IT consultants are requested to report back to 
MBS the value of the contracts they received from ministries under VOR arrangements. 
However, such information, when received by MBS, was not complete. Therefore, 
MBS did not have adequate information on the amount of work that consultants have 
been engaged to do under VOR arrangements. Such information could be used to 
monitor the success of the VOR procurement strategy and to obtain better rates from 
consultants that may receive a significant volume of business from the government. 


The per diem rates proposed by consultants on VOR lists generally remain unchanged 
for the term of the VOR arrangement; however, in some VOR arrangements, 
consultants are allowed to periodically change their rates to reflect market conditions. 
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Recently, MBS has recommended that ministries not request discount pricing from 
consultants and that consultants not submit discounted pricing for assignments. 
According to MBS, the VOR rates that vendors submitted were considered by MBS to 
be the best rates the vendors could offer. However, we noted that MBS does not obtain 
guarantees from vendors that the rates submitted are indeed the lowest rates the vendors 
make available to their major customers, even though such guarantees can usually be 
obtained from vendors for large-volume customers such as the Ontario government. 
Since MBS has not obtained the guarantees, we question why MBS is recommending 
that ministries not negotiate lower prices for their assignments and that consultants be 
prohibited from offering lower rates in certain situations. 





Recommendation 


In order for vendor-of-record arrangements to reflect a fair, open, and 
competitive procurement process that will ensure that ministries receive value 
for money, Management Board Secretariat (MBS) should ensure that: 





e¢ guidelines are strengthened to clarify the process by which consultants are 
selected and that ministries are required to follow a formal selection 
process to give the qualified consultants on a vendor-of-record list equal 
opportunity to bid on government contracts; 


e there is a documented rationale to support any departure from the 
competitive selection requirements of the Directive, authorization from 
Management Board of Cabinet is obtained for all departures, and the 
Directive is updated to reflect the requirements of vendor-of-record 
arrangements; 

e its guidelines to ministries require that larger projects not be subdivided 
into smaller assignments to avoid competition and that ministries ensure 
that consultants assume responsibility for their work by requiring fixed 
deliverables at a firm price; and 


¢ animproved process for collecting information on and Yoel ministry 
use of vendors of record is established. 





In addition, MBS should, whenever possible, obtain guarantees from 
consultants that their per diem rates are the lowest available to their major 
customers. The vendor-of-record list should indicate when these guarantees 
have not been obtained, in which case ministries should be permitted to 
negotiate for better rates. 


Management Board Secretariat Response 


Vendor-of-record arrangements are established through a fair, open, and 
transparent request-for-proposal process whereby proponents responding to 
the request for proposals must meet specified mandatory requirements and 
are then evaluated on their qualifications, experience, understanding of the 
potential consulting assignments, and the rates submitted for the specific 
expertise they are offering to provide. 
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Management Board Secretariat will work with ministries in responding to the 











findings in this report relating to vendor-of-record arrangements and will 
develop and make recommendations to the Management Board of Cabinet that 
the Management Board of Cabinet Directives on procurement be revised to set 
out and clarify the specific rules relating to vendor-of-record arrangements, 
including the establishment of thresholds. In addition, Management Board 
Secretariat will take the necessary steps to address any other issues identified 
in this report. 


In addition, Management Board Secretariat concurs with the recommendation 
of the Provincial Auditor that vendor-of-record guidelines be strengthened and 
clarified to ensure that ministries use the agreements properly and 
appropriately and that vendor-of-record arrangements should seek 
commitments from vendors that they will offer the best prices for services to 
the Ontario Public Service that they offer to similar large organizations within 
Canada. As well, Management Board Secretariat concurs with the 
recommendation that an improved process for collecting information on and 
monitoring ministry use of vendors of record be established. 








CONCERNS RELATING TO THE USE OF 
CONSULTANTS AT MINISTRIES 


For the 2001/02 fiscal year, the ministries at which we audited consulting arrangements 
incurred $293 million in consulting expenditures. ‘The following graph summarizes the 
Ministries’ consulting expenditures over the last five years. 


Combined Annual Consulting Services Expenditures 


at the Six Audited Ministries, 1998-2002 
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Source of data: Ministry of Finance 
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Continuous Reliance on Consultants and 
Assignment Definition 


We found numerous instances in which the Ministries did not comply with the Directive 
requirements that: 


¢ consultants must only be hired on a defined assignment; 


¢ for services ultimately costing more than $25,000, competition must not be avoided by 
awarding the same consultant successive agreements; and 


¢ when appropriate, a transfer of knowledge must occur from consultant to staff to avoid 
a continuous reliance on consultants. 


Specifically: 


e Many of the Ministries depended heavily on the use of consultants, particularly in the 
area of information technology, as the examples in the following table illustrate. 


Consultant Use at Ministries, 2001/02 


Management Board Secretariat 








Health and Long-Term Care (Human Services Cluster) 








Public Safety and Security (Infrastructure Service 
Branch) 


Natural Resources 











Sources of data: Management Board Secretariat, Health and Long-Term Care, 
Natural Resources, and Public Safety and Security 


On average, the per diem rates of these consultants were two to three times the salaries 
and benefits of comparable employees at the Ministries. As an example, the following 
table shows how per diem rates for consultants compare to those of staff at the Ministry 
of Health and Long-Term Care. 
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Comparison of Per Diem Rates 
of Consultants and Equivalent Staff 
at the Ministry of Health and Long-Term Care 


Ministry 
_ Staff 
Per Diem ($) — 


- Consultant 


Classification Per Diem ($) 


Project manager 371 








Database administrator 346 


Business analyst 544 284 


Programmer analyst 


Overall average per diem rates 
for all consultants and staff 








Sources of data: Ministry of Health and Long-Term Care, 
calculations by the Office of the Provincial Auditor 


The Ministries acknowledged that many consultants were engaged to perform day-to- 
day operational duties rather than work on defined assignments. According to the 
Ministries, the reason that they had to extensively rely on the use of consultants was that, 
in recent years, they had had more difficulty in obtaining approval and funding to 
increase the number of their full-time staff and salaries ranges to the levels needed to 
attract qualified candidates. However, such use of consultants was not in compliance 
with the requirements of the Directive that consultants only be hired on a defined 
assignment and that ministries must avoid continuous reliance on consultants. 


We also noted that, at the Ministry of Public Safety and Security, more than 40 
consultants were former ministry employees who had left the Ministry in 2001 and 
were subsequently hired by consulting firms. They returned to the Ministry as 
consultants to perform their previous duties at per diem rates that were much higher 
than their salaries as ministry employees. The Directive states that engaging the services 
of former employees must bear the closest public scrutiny to ensure that no unfair 
advantage exists. Caution must be particularly exercised if less than one year has passed 
since former staff left the employment of a ministry. In this case, we found that the 
majority of the consultants came back within a few days of having left the Ministry. 


We noted that, in many first-time contracts with consultants, the consultant was 
engaged to provide expertise for a short period of time, usually on a three- to six-month 
contract. Once expired, these contracts were extended with little or no change to the 
original deliverables. Competitive selection processes were not always followed, and even 
when they were, they typically resulted in the re-engagement of the original consultants. 
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The following are examples of the effects of these questionable practices that we noted 
from our audit of the Ministries: 


- At MBS, a consulting firm was selected to provide one staff person on a per diem 
basis for the MBS information-technology help desk and site-services support unit. 
The initial contract awarded in April 2000 was for six months at a cost of $50,000. 
At the completion of our audit in March 2002, the consulting firm had received 
three contract extensions, totalling $200,000, without competition and without any 
substantial changes to the original deliverables. 


- In February 1999, a consultant was initially engaged by the Ministry of the 
Environment to provide technical consulting services for three months at a rate of 
approximately $540 per day, with total remuneration not to exceed $25,000. Over 
the following three years, the consultant’s contract was renewed or extended seven 
times without any change in the contracted deliverables, for total remuneration of 
$455,000. The Ministry informed us that, because of the unique experience and 
knowledge of the consultant, it had to continuously rely on the consultant’s services 
to avoid disruption to the Ministry’s operations. In our view, however, the specialized 
nature of the consultant’s work has made it even more important that a process be 
established to ensure appropriate knowledge transfer from consultant to ministry 
staff, since at some point the services of the consultant may no longer be available. 


- Aconsultant was engaged to provide testing services for the Ontario Land 
Information system of the Ministry of Natural Resources. The consultant was 
engaged under three successive contracts, with no change to the original 
deliverables, for services from January 2000 to September 2001, at a rate of over 


$500 per day, totalling approximately $237,000. 


- Aconsultant was engaged by the Ministry of Health and Long-Term Care in 1999 
for an initial term of six months and a price of $96,000. The consultant, who was 
asked to conduct business- and systems-analysis activities for a branch, received 
contract extensions that brought the term to two years. There were no appreciable 
changes to the original deliverables, and the final cost of the assignment after the 
extensions was $360,000. The consultant was eventually replaced with a full-time 
staff member at an annual salary of approximately $60,000. 


Without clearly defined, tangible deliverables that require the consultant to complete 
the assignment by a specified date and at a fixed ceiling price, the assignments may be 
ambiguous, and this ambiguity can often result in the consultants billing over the 


original, agreed-upon amount. 


We also saw little evidence that a formal process was in place to ensure appropriate 
knowledge transfer from consultants to ministry staff to avoid a continuous reliance on 
consultants. 


We were also concerned about the lack of sufficient knowledge and experience within 
the Ministries to carry out senior-level responsibilities and effectively manage the work of 
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consultants. Specifically, we found that the Ministries had to engage senior-level 
consultants at high costs to oversee projects and manage the work of other consultants. 
For example, as at March 31, 2002, the Ministry of Health and Long-Term Care had 
engaged about 15 senior consultants, who charged over $1,200 per day on average, 
compared to salaries of approximately $340 per day, including benefits, that would 
have been paid for senior ministry employees at the equivalent level. 
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Recommendation 


To ensure that its requirements are met in the most economical manner, the 
Ministries should comply with the requirements of the Management Board of 
Cabinet Directive on Consulting Services by: 


e clearly defining proposed assignments, which involves specifying tangible 
deliverables, time frames for completion, and related costs, preferably with 
fixed ceiling prices; and 


* avoiding continuous reliance on consultants and, when appropriate, 
ensuring that a transfer of knowledge occurs from the consultant to staff. 


Management Board Secretariat Response 


The Secretariat agrees that requirements for consulting services must be met 
in the most economical manner possible. The Secretariat has initiated the 
development of a training program for managers on the acquisition and 
management of consultants to ensure that where consultants are used, it is on 
the basis of a strong business case and documentation that includes clearly 
defined details on deliverables, time frames, and related costs. 


The Secretariat also agrees that knowledge transfer is essential in order to 
avoid a continuous reliance on consultants and will take the necessary actions, 
including through ministry training, to further strengthen the knowledge 
transfer requirement. 


Ministry of the Environment Response 


The Ministry acknowledges that consulting contracts should be well defined in 
advance and reliance on consultants monitored. The Ministry is currently 
strengthening its capacity to ensure that all contracts are in compliance with 
Management Board of Cabinet directives. 


To ensure all contracts include well documented assignments, the Ministry has 
developed a procurement checklist for use by all contract managers. The 
checklist clearly outlines the mandatory contract components to ensure 
compliance with Management Board of Cabinet directives. 


Consultants will continue to provide a resource of temporary expertise that 
can be purchased to assist the Ministry to do its business. However, the 





| Ministry's new contract management and quarterly reporting system will 
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ensure regular review by senior management of consultant use across the 
Ministry to monitor ongoing reliance on consulting services and compliance 
with directives. This will allow for early detection of problems and prompt 
management response to any problems that may occur. 


Ministry of Health and Long-Term Care Response 


The Ministry agrees in principle with the recommendation as a long-term 
strategy. However, it should be noted that, in some cases, a more cost-effective 
approach to staffing projects is to bring in outside expertise with the requisite 
skills and experience for the duration of a project, rather than maintain a large 
number of full-time staff to accommodate the specialized resource 
requirements of many of the initiatives. In complying with the government’s 
requirement that ministries effectively manage their staff complement, the 
Ministry must make decisions that enable it to carry out its mandate while 
meeting this commitment. Managers make their decisions based on funding 
allocations and corporate direction. 


It should be noted that most recently in the Human Service Cluster’s [&IT 
Secretariat, the work of the fee-for-service consultants is now being 
undertaken by Ontario Public Service staff. Also, in the Human Services [&IT 
Cluster’s Quality Assurance project, a formal strategy was undertaken to allow 
for transfer of knowledge prior to the termination of the fee-for-service 
contracts. This included documentation of quality assurance work completed 
and processes developed to date, as well as knowledge transfer to recently 
hired full-time staff. 


The Ministry will continue to monitor its use of consultants and raise the 
awareness of program areas to the need to plan for a transfer of knowledge 
where possible and appropriate. 


Ministry of Natural Resources Response 
We will implement the recommendation. 


In concert with the Shared Services Bureau, we will improve procedures to 
more clearly define assignments, including specifying deliverables and fixed 
ceiling prices. 


In the case of the Ontario Land Information system, we are in the process of 
operationalizing the developmental and testing components and transferring 
this knowledge to a core staff team. This will help minimize reliance on 
consultants and ensure skills transfer to ministry staff. 


Ministry of Public Safety and Security Response 


The Ministry has developed tools and processes to ensure that consulting 
assignments are adequately defined, deliverables and time frames specified, 
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and ceiling prices and other terms and conditions clearly documented. The 


introduction of these tools and processes began in December 2001, and 
therefore the results of the improvement to internal controls were not yet 
reflected at the time of the audit. 





To address increasing workloads in the absence of adequate staffing levels, 
particularly in the area of information technology, the Ministry has been 
required to use consultants to continue providing critical technical support to 
front-line service delivery. In addition, a considerable amount of work is 
project-based and requires short-term infusion of highly specialized skills. In 
these cases the Ministry will define the requirements and access consulting 
and other specialized resources through consulting contracts with defined 
deliverables. 


Many information technology (IT) skills are still in very short supply within the 
Ontario Public Service, and we will continue to need to fill gaps with 
consultants. We are moving on a number of fronts concurrently to actively 
address the skill gap— the IT internship program, specific recruitment 
approaches for specialized skills, retention, and succession planning so that 
qualified staff are available to assume critical roles where required. 


With respect to the transfer of knowledge, for defined assignments, it is 
ministry policy to ensure that the transfer of knowledge and expertise by the 
consultant to ministry staff takes place at the conclusion of each consulting 
assignment. We will reinforce that policy. 











Justification for the Use of Consultants 


The Directive states that a ministry must not purchase external consulting services prior to 
considering existing ministry resources. Known resources available from other ministries 
must also be considered. The majority of ministries did not demonstrate that they 
determined whether resources, both within the ministry and in other ministries, were 
available before purchasing external consulting services. Staff at the Ministries expressed the 
opinion that the lack of available skills and resources was obvious and previous attempts to 
identify qualified internal resources had proven to be unsuccessful. 


Proper evaluation and documentation of available resources and anticipated skill shortages 
could help management identify recurring areas where training or hiring staff with the 
required skills would be a more cost-effective long-term strategy for obtaining needed 
services than engaging consultants. 





| 


Recommendation 


To ensure that needed services are obtained in the most economical way 
possible, prior to hiring consultants the Ministries should conduct a proper 
evaluation of available resources both within the Ministries and in other 
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ministries, document the results, assess alternatives to using consultants, and, 
where the services of consultants are considered necessary, justify the 
engagement of consultants. 


Management Board Secretariat Response 


The Secretariat concurs with the recommendation and will improve its 
evaluation of the availability and skills of internal resources before engaging 
outside consultants. In addition, the Secretariat is in the process of 
implementing a strategy to address internal skills requirements, consisting of 
an information technology internship program, specific recruitment 
approaches for specialized information technology skills, retention, and 
succession planning, so that qualified staff are available to assume critical 
roles where required. 


Ministry of the Environment Response 


The Ministry acknowledges the need to improve its documentation to justify 
the use of consultants. The Ministry’s action plan will include training sessions 
to increase awareness among key personnel of the need for better 
documentation and evaluation of the need to hire consultants and a 
requirement that staff complete and document a resource-needs evaluation. 
Technical co-ordinators will be responsible for the review of contracts in order 
to ensure that all proper procedures are followed and documentation is 
complete. 


Ministry of Finance Response 


We agree with the recommendation. In future, we will document why existing 
staffing resources are not able/available to handle the proposed assignment 
and/or any other reasons for retaining outside consultants. 


The Office of the Provincial Controller has been doing this as a matter of 
normal operational practice in the past but has not documented it. The 
evaluation is somewhat subjective and not formalized, but it is based on 
personal knowledge within the management group that is actively involved in 
arriving at this conclusion and includes both the availability of Ministry of 
Finance staff and the financial community in the Ontario Public Service (OPS). 
Due to the current demand for financial expertise in the OPS (for both ministry 
and corporate initiatives such as the Integrated Financial Information System), 
it is extremely difficult to find qualified internal expertise at this time. 


Ministry of Natural Resources Response 


We will implement the recommendations and improve our documentation. 
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In some specialized areas of the Ministry’s business such as water control 
structure engineering and specialized systems development, we have made an 
organizational decision to increase the use of specialized knowledge that 
exists within the private-sector industry as we are not aware of an additional or 
available source of expertise within our own or other ministries. 


Ministry of Public Safety and Security Response 


It is a ministry requirement to conduct evaluations of existing staff resources 
prior to determining that the use of outside consultants is required. The 
Ministry acknowledges that documentation of the results of the evaluation is 
generally inadequate. The Ministry has introduced enhanced internal controls 
related to the acquisition of consultants including a compliance control 
process that will ensure all aspects of the business case for the use, 
acquisition, and monitoring of consulting services are properly documented in 
accordance with Management Board of Cabinet Directives. 











Competitive Selection of Consultants 


We concluded that the Ministries’ procedures for the competitive selection of consultants 
were inadequate to ensure that the Directive and MBS requirements were followed. In 
particular, we found numerous cases where, in the development of multi-million-dollar IT 
projects, the Ministries did not comply with MBS requirements regarding the acquiring of 
consultants from the IT VOR list. 


As indicated earlier, the use of consultants from the IT VOR list is limited to projects with 
an expected cost of $500,000 or less; above that amount, an open tender process is 
required. This requirement is intended to ensure that the most qualified consultants are 
acquired at the best available price, and that all suppliers of consulting services are given fair 
access and treated in an open and transparent manner. 


However, we found that, even where the amounts spent on the projects exceeded 
$500,000, the Ministries did not follow an open tender process. Instead, the Ministries 
elected to engage consultants from the VOR list on successive contracts to work on these 
projects. Some ministries maintained that, by engaging consultants directly from the list to 
do the work, they would have more control over the consultants. However, this practice did 
not comply with the requirement for open tender. 


As indicated earlier, by compensating consultants on a per diem basis rather than with a 
fixed price and on fixed deliverables, the Ministries assumed the risk of consultants not 
delivering their work on time and would often be forced to pay the costs of missed deadlines 
and cost overruns, even when these problems might have been caused by unsatisfactory 
performance and inefficiencies on the part of the consultants. 
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In addition to not following an open tender process, in many instances the Ministries did 
not request written proposals from at least several potential suppliers, did not conduct and 
write up evaluations of proposals, and, when an evaluation was carried out, did not always 
select the most qualified consultants at the lowest cost. 


The following are examples of our observations from our audit of the Ministries. 


Management Board Secretariat 


The first example involves a contract awarded to a consulting firm in April 2000 to assist in 
selecting a vendor to assume responsibility for certain network data functions and 
components. The assignment was defined as having several phases: defining the request-for- 
proposal process; assisting in the development of the request-for-proposal documents; and 
negotiating a contract. We were advised that the contract was awarded to this consulting 
firm by selection from the vendor-of-record list; however, there was no documentation 
available to support the selection of the firm. The rates the consulting firm charged for its 
various consultants ranged from $1,025 to $2,950 per day, and a total of over $1.7 million 
had been paid to the firm as of February 2001. We were advised that the guidelines for this 
VOR list did not include a stated requirement that ministries must document the decision 
process used to select the chosen vendor over the several other vendors on the list. We were 
also informed that this VOR list, in contrast to other VOR lists, did not include a threshold 
amount over which an open tender process is required. In view of the large amount of 
money paid to this firm, it would have been prudent and appropriate for MBS to have 
considered alternative firms in making its selection for this assignment and to have 
documented the rationale for choosing the selected consulting firm. 


The second example involves the selection of an IT consulting firm in April 2000 to provide 
services relating to a management improvement project. We found that only two VOR 
companies were contacted and requested to submit proposals. In addition, there was no 
written evaluation of the proposals providing the reasons for awarding the contract to the 
successful bidder. The firm selected, which had estimated its fees to be $1.4 million, had 
eventually received payments totalling approximately $467,000 when MBS cancelled the 
project. 


Ministry of the Environment 


At the time of our audit, the Ministry was in the process of developing a project called 
Environet. This project was initiated in the summer of 2000 to automate, through the use 
of the Internet, the exchange of information between the public, other stakeholders, and 
the Ministry. The Internet site being developed by the project was to have the following six 
initial components: the Drinking Water Monitoring/Compliance Information System; the 
Air Emissions Inventory System (OnAir); the Hazardous Waste Information Network; the 
Provincial Groundwater Monitoring Information System; eLAB—a system for the delivery 
of environmental information to laboratories and other databases; and SWAT—applications 
and tools used to assist in the enforcement of regulations relating to soil, water, and air. The 
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Ministry's estimate of the cost of developing these components was approximately 


$17.6 million. 
Development work on the Hazardous Waste Information Network and the SWAT 


component of the Internet site went to open tender, which resulted in the Ministry's 
receiving bids for both assignments that were lower than the Ministry’s original cost 
estimates. In particular, the SWAT component was developed at a final cost of $3.5 million, 
compared to an initial ministry cost estimate of $6.2 million. 


The Ministry subdivided the development work for the other four components into 
individual assignments and engaged consultants on a per diem basis from the MBS VOR 
list to do the assignments. The Ministry’s individual cost estimates for three of the four 
components ranged from $1.1 million to $4.2 million, while its estimate for the fourth 
component was $400,000. As indicated earlier, MBS’s VOR-list policy allows for the 
estimated ceiling price of an IT assignment to be up to $500,000 before open tender is 
required. Given that the Ministry’s estimates for three of the components were above the 
$500,000 threshold, the Ministry did not follow the MBS requirement for an open, fair, 


and transparent competitive process for those three contracts. 


According to the Ministry, the reason for selecting consultants on a per diem basis from the 
VOR list was that the business requirements for the four components had not been 
completely defined. By not completely defining the project and undertaking an open 
competition, the Ministry had no assurance that it had obtained the best price and also 
increased the risk of cost overruns on the project. In addition, without a clear understanding 
of what is needed, the ultimate system may not meet the requirements of the users of the 
system. 


We also found the following problems with respect to the engagement of consultants from 
the VOR list for this project: 


¢ In the case of the OnAir component, on two occasions the Ministry had given two 
consultants equal ratings but in each case elected to go with the vendor that charged the 
higher per diem rate. The Ministry could not provide any reason or documentation 
explaining why these vendors, who cost $130,000 more in total, were chosen instead of 
the two that had received equivalent ratings and offered lower per diem rates. 


* In the case of the eLAB component, a ministry selection panel interviewed three firms in 
search of the best one to provide consulting services. The panel rated one of the firms as 
clearly more suitable in terms of both qualifications and price. However, without any 
explanation, the assignment, which was for $316,000, was awarded to the firm rated 
second, which had less experience and submitted a higher bid. The Ministry explained 
that the highest-rated firm was not awarded the assignment because it had already been 
awarded a previous assignment. Subsequently, the firm rated second was awarded 
another contract for the second phase of the same assignment at a price of $375,000. In 
the case of that contract, the Ministry had sent a request for proposals to four vendors, 
but the only bid received was from the firm that had previously worked on the project. 


ee Oe ee 
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Ministry of Finance 


For contracts awarded by the Ontario SuperBuild Corporation—an agency of the Ministry 
of Finance, insufficient care was taken in awarding contracts, as is illustrated in the following 
example. 


When evaluating proposals submitted by consultants for the assignment of providing 
financial advice on the divestment of certain Crown assets, an error was made in 
calculating the scores from the bidders’ evaluation scores. 


In April 2000, two consultants were jointly awarded the contract on the basis of the 
original ranking, and each received $681,000. However, had the scores been correctly 
calculated, one of the two consultants may not have received a contract. 


In addition, these two consultants were not required to sign formal agreements; instead, 
only purchase orders were issued. There was no explicit division of duties and no 
deliverables associated with each purchase order. This makes it difficult to hold 


consultants accountable for their performances. 


In another case, Superbuild Corporation’s poorly defined evaluation criteria led to the 
awarding of a $3-million consulting contract, which was double the price bid by the 
lowest bidder. 


Ministry of Health and Long-Term Care 


We noted that the Ministry did not follow the requirements for an open tender process on 


three multi-million-dollar information technology projects that were under development. 


On each project, the Ministry’s approach was to subdivide the project into smaller 
individual assignments and select consultants from the VOR list on a per diem basis to 


complete the work. Specifically: 


In October 2000, Cabinet approved the establishment of a transfer-payment agency, 
called the Smart Systems for Health Agency, to provide and manage a secure, province- 
wide information network for the exchange of health information among Ontario’s 
health-care providers. The original five-year operating budget for the agency was 
projected at $488 million. This budget was not approved, and the agency has been 
allocated funds on an annual basis. At the time of our audit, the Ministry had yet to 
establish the agency and anticipated that it would be established by September 2002. In 
the meantime, the Ministry had already begun development of the network, taking the 
approach of engaging consultants from the VOR list on a per diem basis. As at March 
31, 2002, in addition to having nine full-time staff on the project, the Ministry had 
engaged more than 65 consultants from various firms at per diem rates that ranged 
from $435 to $2,150. For the 2001/02 fiscal year, the Ministry had spent 
approximately $12.7 million on consultants engaged on a per diem basis. 


In the 1998/99 fiscal year, the Ministry initiated the development of the Integrated 
Services for Children Information System, which is designed to identify and co-ordinate 
information for the protection of potentially vulnerable children. The Ministry's 
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estimated costs for the system were approximately $6 million for initial development and 
$760,000 per year for ongoing support and maintenance. As at March 31, 2002, 
about 50 consultants were working on the project on a per diem basis, and the total 
amount spent was $10.6 million, of which $8.5 million pertained to development costs. 
According to management, the project was about 80% complete, and another 

$3 million was needed to bring the system into full production. 


* The MBS requirement for open tender was also not followed for a project to define and 
implement quality-assurance processes in the area of information technology. The cost of 
engaging consultants on a per diem basis to develop this project was estimated to be 


$5.9 million. 


In these sizeable projects, the Ministry assumed a major risk of consultants not delivering 
their work on time and of cost overruns, even when these problems were caused by a 
consultant's poor performance. It is therefore highly doubtful that the Ministry obtained 
value for money by not adhering to the Directive. 


Ministry of Natural Resources 
In 1998, the Ministry initiated a project called MNR ON-Line to develop and implement 


a strategy for electronic access to the Ministry’s information and services. The Ministry’s 
approach has been to engage consultants from the MBS VOR list on a per diem basis to 
carry out the required work. The work has included assisting in the development of a 
feasibility study, defining the requirements of the project, and implementing the 
infrastructure. As of March 31, 2002, approximately $1.9 million had been spent on this 
project. We have the following concerns with respect to the process for acquiring 
consultants for the project: 


* We would have expected the Ministry to have known that the MNR ON-Line project 
would cost over $500,000 and therefore to have issued an open tender. However, we 
noted that, with the exception of the project manager, the Ministry engaged consultants 
on a per diem basis from the MBS VOR list without going to open tender and thus 
failed to meet Directive and MBS requirements. 


* Although the Ministry had prepared an annual budget for consulting services 
expenditures, it had not estimated the total cost and time required to complete the 
project. 


The lack of open competition and an accurate cost estimate has made it difficult to assess 
whether the final cost of the project is reasonable, making it doubtful whether value for 


money has been achieved. 


Ministry of Public Safety and Security 


Our audit identified a number of instances where the requirements for competitive 
selection were not being followed. For instance: 


ee 
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¢ In August 2000, the Ministry initiated the Information Technology Service 
Management Project to develop an e-service portal that would enable electronic service 
management for the Justice Cluster. The Ministry’s approach to project development 
was to engage a consulting firm from the VOR list to provide per diem consultants to 
define the project’s requirements and carry out the required development work. The 
latest information we had available at March 31, 2002 indicated that the total amount 
spent on the project was $3 million. As indicated earlier, an open tender process is 
required for any IT assignment with an estimated ceiling price of over $500,000. 


We noted that the consulting firm engaged by the Ministry had sub-contracted some of 
the work for the project to another consulting firm on the current VOR list. Had the 
Ministry followed an open tender process, it could have received competing bids 
directly from this sub-contractor and other consulting firms. 


¢ Whenever a ministry plans to select a consultant from the VOR list for an IT assignment 
estimated to cost between $100,000 and $500,000, it is required to send a request for 
resources to at least three vendors from the list. It is then to evaluate the proposals 
received based on prices quoted and the qualifications of the vendors. However, we 
noted that, for many assignments, the Ministry acquired the services of consultants from 


the VOR list without competition. 


It is doubtful whether the Ministry obtained value for money in these projects. 


Recommendation | 


To ensure that consulting services are acquired at the best available price, the 
Ministries should: 





¢ follow the competitive selection requirements of the Management Board of 
Cabinet Directive on Consulting Services and Management Board 
Secretariat; 

* on the basis of its evaluation of the experience, qualifications, and 
submitted bids of all the consultants capable of completing the assignment 
to the satisfaction of the Ministry, select the highest-ranked consultant; and 

¢ adequately document the selection process and retain the documentation 
for use in supporting its decisions. 





Management Board Secretariat Response 


As mentioned earlier, Management Board Secretariat will work with ministries 
in responding to the findings in this report relating to vendor-of-record 
arrangements and will develop and make recommendations to the 
Management Board of Cabinet that the Management Board of Cabinet 
Directives on procurement be revised to set out and clarify the specific rules 
relating to vendor-of-record arrangements, including the establishment of 
thresholds. In addition, Management Board Secretariat will take the necessary 
steps to address any other issues identified in this report. 
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In each of the cases cited, the vendors were selected from a vendor-of-record 
arrangement established through a fair, open, and transparent request-for- 
proposal process. The Secretariat concurs that it must ensure that appropriate 
documentation is completed and retained with respect to its selection process 
for consulting services. The Secretariat will take the necessary steps, including 
appropriate guidance and training for the selection of vendors from vendor-of- 
record arrangements, to ensure that appropriate procedures are followed. 


Ministry of the Environment Response 


The Ministry acknowledges the importance of ensuring that consultants are 
acquired at the best available price. 


The period covered by the consulting services audit covered a time in which 
the Ministry was responding to the Walkerton emergency. The Walkerton 
emergency placed the Ministry under tremendous pressure to deliver 
immediate solutions. Major information technology improvements such as the 


| Drinking Water Management and Compliance Information System were being 


developed as the detailed specifications and consultant requirements were 


_ being defined. 


However, the Ministry will implement the following actions to ensure the 
requirements of the competitive selection process are met in the future: 


e using a standard contract tracking form and procurement checklists in 
order to identify all mandatory steps in the contracting process that will be 
retained in all contract files; 


_¢ training all key staff (technical co-ordinators) to improve ministry 


awareness of the steps required to ensure a competitive process is 
undertaken; 


e dedicating a staff position within the Ministry’s Business and Fiscal 
Planning Branch for procurement and contract monitoring and developing 
strong links to the Shared Services Bureau to ensure that advice on 
procurement issues is readily available; 


¢ establishing and staffing a dedicated, full-time controller position to 
strengthen overall business processes for procurement and contract 
management within the Information Management and Technology Branch 
to assist information technology staff in meeting guidelines and providing 
strategic business advice; and 

e providing quarterly reports to senior management on the competitive 
tendering of contracts to improve the overall monitoring of the use of 
competitive tendering processes for consulting contracts. 


Ministry of Finance Response 


SuperBuild has since adopted the practice of a second-level review of 
competition results using the Shared Services Bureau. SuperBuild also 
accepts and will adopt the recommendation to ensure that evaluation criteria 











194 2002 Annual Report of the Provincial Auditor of Ontario 














be clearly defined prior to the close of the request for proposals and that 4 
written agreements be entered into prior to work commencing. However, we 
note that, in cases of emergency, Management Board of Cabinet directives 
allow work to commence with only a verbal agreement having been made and 
to continue until a written agreement can be put in place. 


Staff involved with project management will be made aware of the 
recommendation by September 2002, via memos and staff meetings . 


Ministry of Health and Long-Term Care Response 


We do agree that hiring consultants to do the work on large projects on a per 
diem basis may not necessarily be the most appropriate methodology. 
However, the intent of the vendor-of-record arrangement appears to be subject 
to interpretation and will need to be clarified in consultation with Management 
Board Secretariat to ensure ministries use vendor-of-record lists more 
consistently. 








Ministry of Natural Resources Response 


We will implement the recommendation. We appreciate your conclusion that 
the Ministry follows a competitive selection process and obtains necessary 
approvals prior to hiring consultants. 


MNR ON-Line is a multi-year initiative to develop and implement electronic 
access to a significant and diverse amount of the Ministry's information and 
services. The Ministry has proceeded with a prioritized list of individual 
projects to achieve its long-term goal. While we believe there is a need to 
harmonize the application of the vendor-of-record policy with the consulting 
services policy, we will improve our project definition, especially if hiring 
consultants on a per diem basis, to avoid the potential of work and cost 
overruns. 





Ministry of Public Safety and Security Response 


The Ministry acknowledges that, in some instances, it has not complied fully 
with the requirements of the Directive on Consulting Services. The Ministry has 
developed processes, tools, and other controls designed to increase 
awareness and understanding and to support consistent compliance with the 
requirements of the Directive by staff across the Ministry. These include the 
strengthening of corporate and divisional controllership capacity, including 
enhanced capacity for the provision of advisory services on financial policies 
and practices. 
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An MBS Agreement with a Consultant 


On March 3, 2000 a forensic accounting firm was engaged by MBS to review certain real- 
estate transactions entered into by the Ontario Realty Corporation (ORC), an agency of 
MBS. There had been allegations that ORC had sold land below market value. The firm 
was selected from the VOR list for estimated fees ranging from $150,000 to $500,000. A 
formal competitive acquisition process was not followed due to the urgent nature of the 
assignment. [he contract between the firm and MBS was signed on March 23, 2000. 


The scope of the original agreement specified that the firm perform a detailed review of 
eight closed sales and a limited review of all remaining properties sold between January 1, 
1998 and March 31, 2000. However, the scope of the transactions to be reviewed 
increased substantially. The scope of the consultant’s services also increased to include civil 
litigation, property investigation, and environmental work. 


An August 2000 memo from the Chief Internal Auditor to the Secretary of MBS dealt 
with the expanding scope of the investigation. It indicated that the current investigation had 
reached the limit of the original terms of reference and that the original contract had been 
entered into to deal with an urgent need for immediate action. He recommended that if 
work was to continue, the March 23, 2000 contract should be ended and new contracts 
entered into to reflect the revised scope and objectives of the ongoing investigative work. He 
further recommended that approvals and authorization for the new contracts should be 
clear and that contract management and payment responsibility should be clarified. 


We were informed that some revisions were made to the original contract, such as a more 
comprehensive detailing of the work to be performed, establishment of a ceiling price and 
date limit, establishment of a joint governance structure with ORC, and establishment of an 
approval-and-authorization process with ORC for invoice approval and payment. However, 
although we were informed that the above described revisions were followed, this revised 
contract was never signed by all parties, and thus the firm was paid for its work without the 
expanded work being reflected in the contract. 


During the 2000/01 fiscal year, billings from the forensic accounting firm, which were paid 
out of ORC’s budget, amounted to almost $6 million. ORC’s board of directors also 
expressed concerns about the expanding scope of the audit and the management of the 
contract. We were informed that, subsequent to the board meeting at which these concerns 
were raised, a joint governance structure was established whose responsibilities included 
ensuring that funds were being expended appropriately. 








: La ee] 
Recommendation 


In order to properly monitor project progress, control project costs, and 
determine the extent to which deliverables are achieved, Management Board 
Secretariat should enter into a new or revised contract with a consultant 
whenever the scope and objectives of the consultant’s original contract are 
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revised, and the new or revised contract should reflect the changes in scope 
and/or objectives. 


Management Board Secretariat Response 


We agree with the Provincial Auditor’s recommendation. We recognize the 
importance of compliance with good contract procurement and management 
practices. When the scope was better defined, we ensured that the specific 
deliverables of the firm were provided in accordance with the informally 
agreed-to revisions. 





Tax Compliance Forms 


In accordance with the internal policy of the Ministries and statements made in the 1996 
Ontario Budget Speech, all bids that vendors submit to the government must include a 
declaration that the vendor's provincial taxes are in good standing. 


The majority of the Ministries did not ensure that potential consultants submitted the 
required tax compliance declaration. Even when consultants did submit the required tax 
compliance declarations, we found that the Ministries did not provide that information to 
the Ministry of Finance to enable verification that the consultants’ provincial taxes actually 
were in good standing. A check by the Ministry of Finance on the tax compliance of the 
consultants in our sample indicated the following concerns: 


e At MBS, two vendors that had received contracts had Ontario addresses not registered 
with the Ministry of Finance, making it impossible to check their tax compliance. Two 
other vendors had tax arrears of approximately $110,000 and $35,000 respectively. 
One of these vendors was also in default for not filing employee health tax returns. 


¢ At the Ministry of Natural Resources, two companies with Ontario addresses, which 
had received contracts of $24,999 and $92,400 respectively, were not registered on the 
Ministry of Finance’s tax roll. 


¢ At the Ministry of Public Safety and Security, two vendors that received contracts were 
in default for not filing corporate tax returns. 








Recommendation 


Prior to engaging the services of a consultant, the Ministries should: 





¢ ensure that the consultant has submitted the required tax compliance 
declaration to confirm that the consultant is in good standing with the 
provincial tax authority; and 

e forward the tax compliance declarations to the Ministry of Finance to 
enable verification that every consultant who submits the tax compliance 
form is actually in compliance. a 
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Management Board Secretariat Response 


The Secretariat will ensure that managers are trained on the need to obtain tax 
compliance forms from the firm prior to awarding a contract. The training will 
advise managers of the required procedures to file the form with the Ministry 
of Finance and maintain a copy in the contract file for reference purposes, as 
required under the Directive on Consulting Services. 


The Secretariat will request that the Ministry of Finance assess the associated 

administrative requirements and develop guidelines, as appropriate, regarding 

verification of a consultant firm’s tax compliance status prior to the completion 
of the assignment. 


Ministry of the Environment Response 


The Ministry recognizes the importance of ensuring consultants’ provincial 
taxes are in good standing. The Ministry will require the use of procurement 
checklists that identify all contract requirements, including the need to ensure 
the submission of a tax compliance form. The ministry training program being 
developed for implementation in the fall will ensure that staff are aware of 
these requirements. 





Ministry of Finance Response 


We agree with the recommendation regarding tax compliance and will pursue 
with Management Board Secretariat the inclusion of the submission of the tax 
compliance declaration as a requirement in the Management Board of Cabinet 
Directive on Consulting Services. 





To strengthen compliance with the Management Board of Cabinet directives on 
consulting services and to create greater awareness of procurement practices 
and requirements for consulting services, we have asked the Manager of 
Modern Controllership Training in our Office to explore with the Shared 
Services Bureau the development of a training course for managers and other 
stakeholders. 





Ministry of Health and Long-Term Care Response 

We agree with the recommendation and will reinforce with program staff the 
requirements relating to the tax compliance form. 

Ministry of Natural Resources Response 


We will implement the recommendation. In concert with the Shared Services 
Bureau, we will confirm all consultants’ tax status. 
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Ministry of Public Safety and Security Response 


The Ministry will consult with the Ministry of Finance to determine what 


process(es) can be established to ensure that the accuracy of tax compliance 


forms can be verified and to identify any related issues and impacts. 














Controls Over Payments to Consultants 


Our audit identified significant weaknesses in financial controls over payments to 
consultants at four of the ministries we audited. The following are our more significant 
observations. 


e At MBS, we identified several instances of invoicing errors and questionable charges for 


which supporting documents could not be provided, as follows. 


- In August 2001, a consultant was paid $14,518 twice for the same invoice. After 


we informed MBS of this overpayment, arrangements were made to recover it. 


- Supporting documents for over $36,000 in charges paid to a consultant in 
December 2000 could not be provided to us by MBS. 


- MBS permitted a consulting firm to significantly increase its rates for two of its 
consulting staff over a short period of time, as follows: 


2 One consultant’s rates increased from $725/day in April 2000 to $1,800/day 


in May 2000 and to $2,600/day in September 2000. 


= The second consultant’s rates increased from $906/day in April 2000 to 
$1,450/day in May 2000. 


We were advised by MBS that these consultant rates were questioned in the fall of 


2000 and that these rates were agreed to in January 2001 after MBS’s review. 
However, no documentation was provided to us to support MBS’s review or its 
rationale for allowing large increases in the consultants’ rates. 


- Ofa total of $90,000 in payments made to a consulting firm for travel expenses 


from December 2000 to May 2001, over $40,000 in payments were made either 
without proper support or with questionable claims (for example, some claims made 
for meal charges were at ten times the rates allowed to government employees). In 


August 2001, an internal audit had identified the problem and, as a result, $7,500 


in reimbursement was obtained from the firm. 


¢ At both the Ministry of the Environment and the Ministry of Public Safety and 


Security, we found numerous examples of payments to consultants that were made 


without a valid contract in place to indicate the terms and conditions of the consulting 
arrangement or exceeded the ceiling price of contracts. In one example, we found that 
payments totalling $760,000 made by the Ministry of Public Safety and Security to a 


consulting firm were not supported by a formal agreement. We were informed that the 
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payments were related to consultants performing day-to-day duties for the Ministry. In 
another example, a consulting firm was engaged by the Ministry to monitor and operate 
computer systems onsite and at remote locations around the province. The arrangement 
continued for seven months after the contract expired, and the firm was paid, in total, 
$200,000 above the contract amount. 


In cases where ceiling prices were exceeded, we found no evidence of prior approvals by 
the Deputy Minister or designate, as required by the Directive. As well, we found no 
evidence of documentation of changes to the terms and conditions of the assignments 
that would have accounted for the amounts charged. 


* The Ministry of Natural Resources did not require that invoices be approved by the 
project manager prior to payment. As a result, it was unclear who was responsible for 
ensuring that invoices were checked for accuracy, corresponded to the terms of the 
contract, and were coded correctly to the proper account. For example, we found one 
case in which the per diem rates charged by a consulting firm were higher than the rates 
quoted. The firm, which received just under $450,000 for the engagement, charged 
hourly rates of $393 and $218 per hour, even though the rates quoted for the 
engagement were $300 and $200 per hour respectively. The Ministry indicated it 
would recover the overpayment after we brought this to the attention of management. 





a | 


Recommendation 


To ensure that all payments to consultants are in accordance with valid 
contracts and made only for work performed, the Ministries should: 





e ensure that approvals at the appropriate level are obtained for consulting 
service invoices submitted for payment; 


* require that payments be made only when there is a valid contract in place; 
and 


° monitor payments for adherence to the agreed-upon price in contracts and 
allow amounts in excess of the agreed-upon price only if those amounts 
are justified, formally agreed to, and accompanied by proper approval. 


Management Board Secretariat Response 


The Secretariat is tightening controls on payment processes through 
implementation of its revised delegation of authority framework and through 
training on the associated payment-process-verification controls (in 
conjunction with the Shared Services Bureau). As part of its procurement 
training program, managers will be reminded of the need to ensure that 
payments are only made based on the stipulated terms of the contract and in 
compliance with applicable directives. In addition, the procurement training 
program will address the need for managers to ensure that consultants’ 
contractual terms align with the Ontario Public Service Directive limits for 
expense reimbursements and that managers make CONS IaETS aware of, and 


make consultants adhere to, these limits. 
fea ed 
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Ministry of the Environment Response 





The Ministry acknowledges that improvements need to be made to ensure that 
all payments made to consultants are legitimate and made only for work 
performed. 


The Ministry's financial delegation of authority and policies are being revised 
and streamlined for more efficient and controlled procurement practices, and 
training sessions will be held for all staff involved in procurement. 





The Ministry has met with Shared Services Bureau finance staff to ensure that 
payments are only made when a valid contract is in place and payments do not 
exceed the ceiling price. 





The Ministry’s new internal tracking and reporting system for consulting 
contracts will also ensure that payments are properly managed and tracked 
against ceiling prices. 








Ministry of Natural Resources Response 


We will implement the recommendation. 





It is our normal practice to require project managers to review and approve 
invoices for payment. We will reiterate this requirement to our staff. 


In the case of the per diem variance, this contract was completed very much to 
our satisfaction, within the total price quoted. We will, however, recover the per 
diem overpayment. 





Ministry of Public Safety and Security Response 


As mentioned previously, commencing in December 2001, the Ministry 
introduced a number of processes and tools to improve its control and 
management of contracts. Measures include the strengthening of divisional 
controllership capabilities as well as the development of a contract-tracking 
database to assist in controlling contract expenditures against established 
“ceiling amounts. 














Managing and Controlling the Use of Consultants 


The Directive states that ministries must be prepared to provide a report on their ability to 
effectively manage the process of planning for, acquiring, and managing consulting services. 


Specifically, the report is to include: 


¢ a detailed summary of the use of consultants during the reporting period and a review 
of ministry practices with respect to each principle and requirement of the Directive; 


¢ information on all agreements for which a firm ceiling price was not established; 





Consulting Services 201 





| VFM Section 3.06 | 


| VFM Section 3.06 








¢ information on all agreements in which an increase in ceiling price occurred; 
e information on all follow-on agreements that were not tendered; and 


¢ information on all agreements with a ceiling price of between $25,000 and $99,999 in 
which non-competitive acquisition procedures were used. 


The majority of the Ministries had not established adequate systems and procedures to. 
monitor the use of consultants by program area to ensure that consulting services were 
acquired economically and to identify areas where management practices and controls 
needed to be improved. 


If such information could be readily produced, it would allow the Ministries to undertake 
overall monitoring activities such as: comparing the estimated and actual costs of 
assignments to determine if their estimating procedures needed to be improved; assessing 
whether or not adequate explanations were given for assignments that were not acquired 
competitively; and establishing whether or not justification existed for increases in 
assignment costs over the contract ceiling price. 


In view of the hundreds of millions of dollars spent on consulting services, the Ministries 
would benefit by having regular monitoring information on these significant expenditures 
and their performance in meeting the requirements of the Directive, in that such 
information would allow for timely corrective action. 








Recommendation 


To help ensure that consulting services are acquired, managed, and controlled 
appropriately and economically, the Ministries should establish an adequate 
system for maintaining management information on the use of and payments 
to consultants by the various program areas. The information should be used 
for monitoring the effectiveness of the use of consultants by the Ministries and 
for identifying areas where management practices need to be improved. 





Management Board Secretariat Response 





The Secretariat will identify and implement improvements to current 
management information systems to ensure that we have appropriate 
information on the use of consultants and can assess where management 
practices may need improvement. 








Ministry of the Environment Response 


The Ministry acknowledges the need to properly manage and control the use of 
consultants. A key element of the Ministry’s action plan to improve consulting 
contract management will be the introduction of a new tracking and reporting 
system for all consulting contracts. - | 
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Quarterly reports, containing contract information by division and program 
area, will be prepared for senior management for control and monitoring 
purposes. The reports will provide early alerts to problems in order for 
immediate follow-up and resolution. 


Ministry of Health and Long-Term Care Response 
Agreed. 


The Ministry’s current consultant procurement process includes preparation of 
a business case that includes an assessment of the most appropriate 
acquisition process. Documented justification is also required for any 
amendments to initial contracts that will increase the original ceiling price. 


In its training sessions on procurement and the educational tools provided to 
program staff, the Ministry does review the best practices that should be used 
to manage and control consultants. The Human Services Cluster has initiated 
an overall monitoring process that reviews all its consultant contracts on a 
quarterly basis. 





The Contract Management and Reporting System is primarily a reporting tool 
to capture the information noted in the Provincial Auditor's findings. The 
Ministry will review the feasibility of using the system as the master system for 
recording all contracts and for doing periodic reviews. 


Ministry of Natural Resources Response 


We will implement the recommendation in concert with the Shared Services 
Bureau and establish a more systematic approach to collecting this 
information. We will enhance contract administration training. The Shared 
Services Bureau is currently working on enhancing a contract administration 
manual to be used in conjunction with the training and as a tool for ongoing 
reference by project managers. 


Ministry of Public Safety and Security Response 


The Ministry accepts this recommendation and, prior to the commencement of 
the audit, was nearing completion on the development of a contract-tracking 
database to support the management, control, and reporting of consulting 
contracts. Completion of the database was delayed due to the recent labour 
dispute in the Ontario Public Service. The Ministry also took this opportunity to 
make significant improvements to meet corporate information technology 
architecture requirements and further enhancements to the client database. 
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Post-assignment Evaluations 


A written performance appraisal of the consultant should be prepared after the completion 
of an assignment. Performance appraisals are required in the case of contracts with vendors 
of record. Performing the appraisal enables an assessment of: the quality of the work; 
whether value for money was obtained; and the suitability of the consultant for future work. 


We found that the Ministries generally did not prepare formal post-assignment evaluations 
of consultants, and the evaluations that were prepared lacked sufficient detail to determine 
aspects of the consultant’s performance, such as whether it was satisfactory, met all milestone 
dates, and was within budget. In addition, prior to awarding new contracts to companies, 
past performance was not verified to ensure that previous contracts had been satisfactorily 
carried out. 


For example, in the development of the Hazardous Waste Information Network (a 
component of the Environet project) at the Ministry of the Environment, a consultant was 
hired at a price of approximately $270,000 to prepare a request for proposals. We were 
informed that, after an assessment of the consultant’s work, the Ministry had to spend 
$250,000 to have the work done again. However, since the assessment was not performed 
on a formal basis, the Ministry had no documentation of the assessment. 





Recommendation 


To better ensure that value for money is received from consultants, the 
Ministries should ensure that all major consulting projects are formally 
evaluated upon completion and that the results are documented for use in 
determining the suitability of the consultants for future work. 


Management Board Secretariat Response 


Management Board Secretariat concurs that documented evaluations of 
consulting assignments should be completed. The instructions for using a 
vendor-of-record arrangement include the requirement to complete a 
performance evaluation for each consulting assignment. Management Board 
Secretariat will take the necessary steps to ensure that this is done in the 
future. 


Ministry of the Environment Response 


The Ministry acknowledges that post-assignment evaluations are a key element 
of good contract management. The Ministry will ensure that a standard post- 
assignment evaluation form is completed for all contracts. 


The Ministry’s new internal tracking of consulting contracts will ensure that the 
requirement for a post-assignment evaluation is identified, and it will be the 
responsibility of the technical co-ordinators to ensure that contract files are 








not closed until this final step is completed. 
eS ee ee ee 
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Ministry of Finance Response 


We agree with the recommendation and will implement an evaluation process 
to assess the consultants’ performance and degree to which the assignment 
objectives were achieved. This evaluation will be placed in the consulting file at 
the completion of the contract. 











The Ministry will approach the Shared Services Bureau to determine the 
feasibility of developing a procedure whereby all consultants’ performance 
evaluations are filed centrally and made available to the Ministry. 


Ministry of Health and Long-Term Care Response 


Agreed. In fact, the Ministry’s guidelines in both its handbooks and on the self- 
help Web site suggest that a post-performance appraisal is a best practice. The 
Ministry will continue to emphasize this point to program managers in its 
procurement training. 
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Ministry of Natural Resources Response 


We will implement the recommendation in concert with the Shared Services 
Bureau, including making modifications to the Performance Evaluation form. 


Ministry of Public Safety and Security 


As has been mentioned previously, the Ministry has developed a contract 
compliance process to help staff follow the requirements of the Directive on 
Consulting Services and to ensure appropriate documentation is maintained 
for all consulting engagements. A key component of this process is the 
completion and documentation of a post-engagement review to assess the 
performance of the consultant against contract deliverables. 








OTHER MATTER 
ONTARIO REALTY CORPORATION 


In the course of our attest audit work on the Ontario Realty Corporation (ORC)—an 
agency of MBS—we reviewed contracts pertaining to the acquisition of creative 
communications consultants. The acquisition, management of, and controls over creative 
communications consultants fall under a different Management Board of Cabinet directive 
than the Management Board of Cabinet Directive on Consulting Services (see the 
Background section for the specific consulting services covered under the Directive) and 
therefore are not within the scope of this value-for-money audit. However, our work on the 
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ORC revealed similar issues to the issues raised in our value-for-money audit, and for that 
reason we provide the following summary of our findings. 


During the 2000/01 fiscal year, many consultants were engaged by ORC to deal with 
criticisms by members of the Legislature of past real-estate transactions. Of these consultants, 
the communications firms whose contracts we reviewed billed ORC a total of $1.4 million, 
with one firm billing over $1 million. 


The largest firms reviewed were chosen through standing agreements with Management 
Board Secretariat’s Advertising Review Board (ARB). These firms were selected based on a 
public-relations-company competition administered by the ARB. Firms were publicly 
invited to compete based on ORC’s need for a public-relations agency that could provide 
full service to ORC. 


One standing-agreement contract per company was signed. These contracts did not include 
a ceiling price, as the standing agreements required that ORC and the consultants enter 
into a “letter of agreement” for each specific project or task. The letter of agreement is 
intended to establish the services to be provided, the timetable, the contract price, and such 
other terms and conditions to which ORC and consultants agreed. 


We observed that letters of agreement were frequently not completed or signed. For 
example, one firm’s standing agreement was signed October 25, 1999. However, the first 
letter of agreement was not signed until September 15, 2000, by which time over 
$630,000 in services had been billed by the firm. 


Despite the fact that this firm billed and was paid over $1 million, we found only five letters 
of agreement with this firm, which established maximum billings aggregating to only 
$210,000. Where letters of agreement were completed, we noted that the scope of the 
assignment was often very general in nature, defined by phrases such as “providing advice 
on a variety of issues impacting ORC” and “providing external and internal 
communications advice.” 


We also observed that invoices submitted by the consultants often lacked sufficient detail to 
determine whether the amount invoiced was reasonable. 


A review of communications consultant billings revealed that consultants billed for work in 
all aspects of ORC’s corporate communications. These included preparing annual reports 
and business plans, writing briefing notes, assisting in communications in connection with 
ORC’s change in agency status, and developing Web sites. Management Board directives 
discourage users of consultants from becoming overly dependent on the consultants’ 
services. 


ORC management informed us that during the 2000/01 fiscal year, the organization was in 
the media spotlight and dealing with criticisms at Queen’s Park over past real-estate 
transactions. ORC staff indicated that in this environment, it often had to act quickly and 
therefore was not always able to fully comply with formal contract procurement and 
management practices. 
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Recommendation 


The Ontario Realty Corporation (ORC) should ensure that justification for 
hiring communications consultants is documented. A needs analysis should 
include the costs and benefits of hiring the consultant and alternatives 
considered such as the use of ORC staff. In order to reduce costs, ORC should 
also attempt to lessen its dependency on communications consultants by 
performing as much work as possible in-house. 


To ensure proper contract management with communications consultants, 
ORC should comply with the terms of the Advertising Review Board standing 
agreements that require that letters of agreement be entered into for each 
individual assignment. ORC should also ensure that, for each agreement, the 
project scope and deliverables are clearly defined in sufficient scope and detail 
to permit the effective management of the contracts ag to ensure objectives 
have been met. 


Invoices submitted by communications consultants should provide sufficient 
information to allow staff responsible for invoice approvals to determine 
whether the service has been rendered and that the amount invoiced is 
reasonable. 


A gency Response 


ORC agrees with the recommendation. In future, corporate procurement 
procedures will be used for all purchases of goods and services. 
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MINISTRY OF NATURAL RESOURCES 


3.07—Ontario Parks 
Program 





BACKGROUND 


The Ontario Parks Program (Program) of the Ministry of Natural Resources is responsible 
for managing provincial parks and protected areas in support of the Ministry’s vision of 
sustainable development of natural resources and its mission of managing such resources for 
ecological sustainability. 


The primary objectives of the Program are to protect natural resources, provide recreational 
opportunities, develop tourism, and enhance appreciation of the province’s natural and 
cultural heritage. The major responsibilities of the Program include enforcing legislation, 
operating park visitor services, park planning, and maintaining information systems. The 
Program is also responsible for administering the Endangered Species Act, which includes co- 
ordinating the development of a province-wide strategy for species at risk of extinction and 
the preparation of recovery plans for individual species at risk. 


Public land in Ontario can be designated as a provincial park under the Provincial Parks 
Act, and the Act states that “the provincial parks shall be maintained for the benefit of 
future generations.” Public land can also be designated as a conservation reserve under the 
Public Lands Act for the purpose of protecting natural areas and permitting traditional 
public land uses, but commercial activities such as timber harvesting and mining are 
prohibited. The Program has policy responsibility for conservation reserves, but the 
Ministry's Field Services Division, through local district offices, performs other activities such 
as site management and legislative enforcement. The number and area of Ontario’s parks 
and reserves regulated at the time of our audit were as outlined in the following table. 
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Provincial Parks and Protected Areas 


Provincial parks 


Conservation reserves 





“AS part of Ontario’s Living Legacy (a comprehensive 
program for the protection of natural resources announced 
in March 1999), an additional 192 conservation reserves 
have been approved and receive interim protection but 
have not formally been established by regulation. 


Source of data: Ministry of Natural Resources 


Depending on the intended use, provincial parks can be deemed as either operating 
parks—whereby services and facilities are made available to the general public—or non- 
operating parks—which are specific areas designated as park land but not developed for 
recreational use. Many non-operating parks are established for educational and scientific 
purposes. 


The parks provide over 19,000 car-accessible campsites and 7,000 wilderness campsites, as 
well as day-use areas, picnic facilities, and a number of visitor centres and heritage museums. 
The Ministry estimates that annually the provincial park system creates 14,000 person-years 
of employment and contributes $390 million to the Ontario economy. 


In February 1996, the government approved a new business model for the protection and 
management of the provincial park system. The Minister of Natural Resources was given the 
administrative authority to set provincial park fees, establish a board to provide advice on the 
management of provincial parks, and deposit all provincial park revenue into a Special 
Purpose Account dedicated to Ontario Parks’ expenditures. 


For the 2001/02 fiscal year, the Ministry’s funding for the Program was approximately 
$55 million, of which $41 million was funded from the Special Purpose Account. Ministry 
capital spending on the Ontario Parks infrastructure totalled an additional $15.6 million. 


AUDIT OBJECTIVES AND SCOPE 


The objectives of our audit of the Ontario Parks Program were to assess whether the 
Ministry had adequate procedures in place to: 


* ensure compliance with the legislation and ministry policies that are designed to ensure 
that park resources are sustained to benefit future generations; 
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* measure and report on the Program’s effectiveness in managing the public use and 


ecological sustainability of provincial parks; and 
* ensure that resources were managed with due regard for economy and efficiency. 


The criteria used to conclude on our audit objectives were discussed with and agreed to by 
ministry management and related to systems, policies, and procedures that the Ministry 


should have in place. 


Our audit was performed in accordance with the standards for assurance engagements, 
encompassing value for money and compliance, established by the Canadian Institute of 
Chartered Accountants, and accordingly included such tests and other procedures as we 
considered necessary in the circumstances. 


Our audit included a review of the activities of the Ministry’s Internal Audit Services 
Branch. However, we did not reduce the extent of our audit work, as the Branch had not 
issued any recent reports on the administration of the Ontario Parks Program. The Branch 
did carry out an audit of Ontario Parks’ Special Purpose Account, which we reviewed. We 
incorporated any relevant concerns arising from that review into our audit procedures. 


OVERALL AUDIT CONCLUSIONS 


Overall, we concluded that, in many respects, the Ministry did not ensure compliance with 
the legislation and policies designed to ensure the sustainable use and development of park 
resources. Furthermore, the Ministry did not have adequate procedures in place to measure 
and report on the effectiveness of the Ontario Parks Program in maintaining the parks 
system for the benefit of future generations. Specifically, we noted the following: 


¢ The Ministry had management plans in place for only 117 of the 277 provincial parks. 
Such plans are essential if animal and plant life resources are to be managed and 
protected. We noted instances where inadequate planning and a lack of action resulted 
in uncontrolled wildlife growth and habitat destruction that threatened the 
sustainability of other species. In addition, in over half of the non-operating parks we 
reviewed, the Ministry had not prepared the required inventories of animal life, plant 
life, geology, and other natural features. This information is a critical first step in 
ensuring fragile park resources are properly managed. 


¢ The Ministry did not have an overall strategy in place to manage species at risk of 
extinction in the province even though the Endangered Species Act has been in force 
since 1971. Of the 29 species deemed by regulation to be at risk, only five had recovery 
plans in place. Two species of butterfly and a species of rattlesnake, all of which did not 
have a recovery plan in place, can no longer be found in Ontario. In addition, there 
were 35 species that were not protected by regulation under the Act, even though some 
of these species had been designated as endangered as long ago as 1984. 
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¢ Although the Ministry had identified a set of outcomes related to its objectives, it had 
not defined performance measures to assess the overall effectiveness of the Program. The 
measures it did have in place did not specifically relate to an evaluation of the ecological 
sustainability of provincial park resources. 


In addition, we noted a number of instances where procedures to ensure due regard for 
economy and efficiency needed to be improved. For instance: 


¢ Customer service standards were not met for the Computer Reservation and 
Registration Accounting System (operated by a private service provider), in that over 
65% of our sample telephone calls were not answered either because of a busy signal or 
because we were put on hold for 15 minutes (after which time we hung up the 


telephone). 


¢ According to the Ministry, the majority of the existing capital infrastructure, including 
buildings, roads, bridges, docks, and water distribution systems, is between 20 and 45 
years old and is approaching the end of its useful life or is beyond its useful life. The 
Ministry had a backlog of capital infrastructure projects to complete, resulting in the 
deterioration of Ontario Parks’ infrastructure. 


DETAILED AUDIT OBSERVATIONS 


COMPLIANCE WITH POLICIES AND 
PROCEDURES 


Managing Parks for Ecological Sustainability 


Ministry policy requires that park management plans be prepared for each provincial park 
and reviewed and updated every 10 years. These plans provide strategic direction for the 
management of the resources within individual provincial parks to achieve park objectives 
related to protection, heritage appreciation, recreation, and tourism. Central to the 
management plan process is the requirement that detailed inventories be completed of the 
park’s cultural heritage, wildlife, recreational resources, geology, and other natural features. 
We reviewed the Ministry's park management plans and monitoring process and had the 
following concerns: 


¢ Of the 277 provincial parks that existed at the time of our audit, only 117 had 
management plans in place. Furthermore, 68 of the plans that were in place had not 
been reviewed for more than 10 years, with the average age of these plans being 15 
years. Without a current management plan in place, there is no overall strategy to 
manage park resources to ensure that the environment is protected and that park 
resources are optimally used. In that regard, we noted that another Canadian 
jurisdiction is required to review park management plans every five years because 
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ecosystems are vulnerable to outside influences and can rapidly change or deteriorate. 
For example, we noted that it took over 20 years to prepare a management plan for one 
eastern Ontario park. The lack of strategic direction for resource management during 
that time resulted in uncontrolled wildlife population increases that caused severe 
damage to vegetation in the park and the destruction of habitats that sustain rare 
species. 


In parks where management plans existed, we noted that the Ministry did not ensure 
that the plans to address resource issues within the park were implemented in an 
effective and timely manner. For example, at one southwestern Ontario park the 
Ministry did not act on problems associated with the destruction of vegetation by 
wildlife, thus threatening the sustainability of other species in the park. Specifically, the 
park can sustain a population of 65 to 75 deer, but it currently has over 130 deer. 
Scientific research carried out by a university determined that this has resulted in a 
permanently altered ecological state in that park. The research noted that an 
endangered bird species was disappearing since its preferred nesting sites were not 
regenerating within the park because of habitat destruction. The Ministry has 
determined that loss and degradation of habitat is the single biggest cause of wildlife 
species extinction. 


The Ministry did not maintain a provincial inventory of protected resources, did not 
know how many inventories were still to be completed, and did not know when existing 
inventories had been prepared. We also noted that 57% of our sample of non-operating 
parks did not have a detailed inventory of plants, animal life, geology, and other natural 
features. Complete inventory information is the cornerstone of preparing an effective 
management plan and successfully managing the resources within provincial parks. The 
lack of such information reduces the Ministry’s ability to ensure the sustainable use of 
park resources, and management practices, in the absence of this information, may 
result in the degradation of ecologically sensitive areas. 


In addition to preparing inventories to help monitor the health of an ecosystem, many 
jurisdictions identify an indicator species within protected areas. Focusing on an 
indicator species and its habitat facilitates the monitoring of ecological conditions and 
the determination of any corrective action that may be required. At the time of our 
audit, the Ministry was in the process of identifying indicator species. 


The Ministry uses implementation plans to provide direction for more specific projects 
and activities to manage and protect park resources and to identify the financial, 
material, and human resources needed for the preparation of annual work plans. 
However, the Ministry did not track the overall progress of the implementation plans 
that have been developed. Overall monitoring of implementation plans would help 
determine how well the Program is achieving the planned objectives. It would also 
enable the Ministry to provide a consistent basis for determining the allocation of 
resources to specific projects and activities. The Ministry advised us that human and 
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financial resources are scarce and must be allocated among competing priorities. As a 
result, carrying out implementation plans is becoming more difficult, and the Ministry 
risks not adequately protecting park resources. 








Recommendation 


To help ensure that provincial park resources are protected and maintained at 
sustainable levels, the Ministry should: 


¢ complete management plans for all parks and review existing plans on a 
more timely basis; 


¢ complete and monitor the required resource inventories for all parks; 


e develop procedures, such as the selection and monitoring of indicator 
species, to help evaluate and report on the sustainability of park 
ecosystems; and 


e conduct a province-wide risk analysis that will result in financial and 
human resources being directed to the most critical areas and ensure that 
the related implementations are effectively monitored. 


Ministry Response 


The Ministry accepts the validity of the concerns expressed regarding the need 
to develop management plans, inventories, sustainability indicators, and a 
province-wide risk analysis and is fully committed to undertake these on a 
priority basis. 


Management plans and measurement tools are important, but are not 
necessarily the only means to secure ecological integrity. The designation of 
land as a park or conservation reserve ensures that the activities that pose the 
greatest threat to natural ecosystems, including settlement and industrial uses, 
are prohibited or take place under tightly controlled conditions. A range of 
legislation, regulations, and policies are used to ensure that parks are 
protected. 


The management planning process entails a multi-stage public consultation 
process. The Provincial Auditor’s report notes the case of the southwestern 
park experiencing an over-population of deer. The ongoing challenge to the 
Ministry is to balance the objectives of resource protection with the sometimes 
conflicting public sentiments and values. In this particular case, a successful 
deer herd reduction plan has been implemented that is consistent with the 
principles of ecological sustainability and has the support of the local 
Aboriginal community and other stakeholder groups. 


The Ministry accepts the Provincial Auditor's findings with respect to the 
required resource inventories. The Ministry will assess the status of its existing 
resource inventories and set priorities for completing inventories with available 
staff and financial resources. 
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In respect of evaluating and reporting on the sustainability of park ecosystems, 
the Ministry accepts the Provincial Auditor’s findings and is currently 
developing a framework to define indicators. This framework will address 
various components of the environment including land, water, and see as 
potential indicators of sustainability. 


The Ministry agrees with the findings of the Provincial Auditor with reapect to 
implementation plans. As is the case with management plans, the Ministry will 
ensure that available financial and human resources are directed to the most 
critical areas. 








Species at Risk of Extinction in Ontario 


The province enacted the Endangered Species Act in 1971 to provide for the conservation, 


protection, restoration, and propagation of species of flora and fauna of the province of 
Ontario that are threatened with extinction. In 1999, Ontario’s Living Legacy program, 
which was established to provide a long-term program of natural-heritage protection in the 
province, inaugurated the Species at Risk Project. Although the project is responsible for 
species at risk throughout the province—not just in parks—it is administered through the 
Ontario Parks Program. The project provides advice on the development of regulations, 
policies and guidelines, recovery plans, partnership agreements, and field projects across 
Ontario. The goal of the project is to maintain, enhance, or restore the ecological 
sustainability of ecosystems to achieve the conservation, protection, or recovery of species at 
risk, and the existence of healthy populations of all native species that make up and 
contribute to the biological diversity of these ecosystems. For the 2001/02 fiscal year, the 
project received base funding of $187,000 from the Fish and Wildlife Program and 

$2.2 million from the Ontario’s Living Legacy initiative. 


The Ministry’s Committee on the Status of Species at Risk in Ontario (COSSARO) 
conducts reviews of available data on those Ontario species that are to be considered for 
evaluation by the federal Committee on the Status of Endangered Wildlife in Canada 
(COSEWIC). In 1996, in conjunction with its federal and provincial counterparts, the 
Ministry signed the Accord for the Protection of Species at Risk and committed to a 
national approach for the protection of such species. In signing the agreement, the Ministry 
recognized COSEWIC as the source of independent advice on the status of species at risk. 
Approximately 47% of all nationally designated species at risk are in Ontario, and 80% of 


these species are native to southwestern Ontario. 


We reviewed the Ministry’s efforts to protect and restore species at risk and had the 
following concerns: 


¢ The Ministry did not have an overall strategy for species at risk even though the 
Endangered Species Act has been in force since 1971. Such a strategy would set out the 
principles, goals, and general policies that would enable the Ministry to develop a 
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structured program for species at risk. At the completion of our audit in January 2002, 
we were informed that the Ministry was in the process of developing such a strategy. 


¢ As of January 2002, the Ministry did not have recovery plans in place for most of the 29 
species protected under the Endangered Species Act. Some of these species were 
designated as endangered by regulation under the Act as long ago as 1973. The 
following table specifies the recovery plan status of the 29 species. 


Recovery Plan Status for Endangered Species 


: Recovery: 
_ Plan Status 











Completed 
Draft stage 7 were designated prior to 1995 


Not started 7 were designated prior to 1978 


Source of data: Ministry of Natural Resources 


Without recovery plans, it is difficult for the Ministry to effectively manage species at risk 
to ensure both the continued existence of such species within the province and their 
future sustainability. For example, COSEWIC has determined that three species that 
were designated as endangered by regulation under the Endangered Species Act can no 
longer be found in the province. They are the Timber Rattlesnake, regulated in 1973, 
and two butterfly species regulated in 1990—the Frosted Elfin and Karner Blue 
butterflies. Two of these species never had a recovery plan in place, and the recovery 
plan for the remaining species was only in the draft stage. 


¢ Both COSSARO and COSEWIC had determined that, in addition to the 29 species 
already identified, another 31 species in Ontario were endangered, and these species 
were recorded on a backlog list, yet to be regulated. Some of these species were 
designated as endangered as long ago as 1984. Delays in addressing the needs of these 
endangered species may result in their extinction in the province. 












Recommendation 


To properly manage species at risk and to help sustain and increase 
endangered populations, the Ministry should: 


e develop an overall strategy to provide for the conservation, protection, 
restoration, and propagation of species at risk; 
e clear up the backlog for regulating identified endangered species; and 


¢ prepare and implement recovery plans to help prevent species from 
becoming extinct in the province. 











nnn 
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Ministry Response | 


The Ministry agrees with the Provincial Auditor’s findings with respect to the 
need for an overall strategy for species at risk. The Ministry completed a Draft 
Strategy for Species at Risk in Ontario and circulated it for internal review 
within the Ministry in February 2002. Pursuant to internal approval the 
document will become available for external review. 


The Ministry accepts the recommendation of the Provincial Auditor with 
respect to clearing up the backlog for regulating identified endangered 
species. It is important to recognize that the provincial Endangered Species 
Act is not the only mechanism available to provide protection. Legal and/or 
policy-level protection is also provided to species and habitats under other 
provincial and federal statutes, including: the Fish and Wildlife Conservation 
Act, the Provincial Parks Act, the Crown Forest Sustainability Act, the Planning 
Act, the federal Fisheries Act, and the federal Migratory Birds Convention Act. 


Additional funding made available under Ontario’s Living Legacy has enabled 
the Ministry to partially address the backlog on a priority basis. 


The Ministry agrees with the Provincial Auditor’s finding that recovery plans 
should be developed and action plans implemented to help prevent species 
from becoming extinct in the province. In keeping with the 1996 Accord for the 
Protection of Species at Risk in Canada, the Ministry will develop recovery 
plans for Ontario’s endangered and threatened species and is implementing a 
broad range of activities to manage species at risk, including stewardship, 
research, monitoring, habitat restoration, and awareness. 


Ontario is home for more species at risk than any other jurisdiction in Canada. 
Given the workload associated with providing for the protection, conservation, 
and recovery of these species, Ontario has initiated the development of a 
proposed evaluation process, “Assessment of Conservation Priorities”, which 
will be used in determining where to allocate resources in a manner that 
addresses the most urgent conservation needs. 


Ontario will be the lead for developing recovery strategies for endangered and 
threatened species (for which Ontario has jurisdictional responsibility) that 
occur solely in Ontario and will co-lead the development of recovery strategies 
where a species occurs in Ontario as well as in other provinces or territories. 





Recovery strategies will be developed for individual species where the habitat 
requirements and the specific limiting factors warrant a single species plan 
and on an ecosystem basis in which the focus will be on recovering the habitat 
of multiple species that may be endangered, threatened, or at other levels of 





risk. 
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Enforcement Activity 


The Provincial Parks Act stipulates that no person shall remove, damage, or deface any 
property of the Crown or damage or deface any relic, artifact, or natural object or any site of 
archaeological or historical interest within a provincial park. In addition, to ensure that the 
Ontario Parks Program can continue to contribute to the enjoyment of present and future 
park visitors and to protect park ecosystems, the Act regulates human activities within the 
park such as mining, fishing, camping, and recreational pursuits. The Ministry employs 
approximately 250 park wardens to enforce the legislation by conducting general deterrent 
patrols to protect park property and natural resources. An additional 150 ministry staff, 
including all park superintendents and assistant superintendents, have the power and 
authority of a park warden. Park superintendents are responsible for ensuring that the level 
of enforcement activity carried out by park wardens is adequate. Within park boundaries, 
park superintendents and wardens have all the power and authority of members of the 
Ontario Provincial Police. 


We visited a number of provincial parks and reviewed the overall enforcement activities. We 
also reviewed the enforcement monitoring reports maintained at these parks and, from 
either interviews or surveys, we received responses from over 90% of the 59 provincial park 
superintendents. Based on this work, we concluded that park resources were not adequately 
protected and that enforcement efforts needed to be improved. 


Overall, three-quarters of the park superintendents who responded stated that the parks 
that they are responsible for had not been adequately maintained to ensure that natural 
resources were protected. Specifically, with respect to operating parks, almost 70% of the 
park superintendents indicated that the parks were not being effectively patrolled and that 
the Ministry’s minimum operating standards relating to enforcement were not being met 
due to inadequate funding, insufficient staff, and substandard equipment. As a result, park 
wardens noted that they were not able to adequately carry out enforcement activities. There 
were also a number of customer complaints regarding the lack of enforcement staff in 
operating parks. For example, park campers expressed concern about their safety in the 
evening hours and expected major campsites to be patrolled by park wardens. 


Enforcement activity was also inadequate in non-operating parks. Most park 
superintendents noted that insufficiency of resources was the reason for this inadequate level 
of enforcement activity. Over half the superintendents stated that staff visit non-operating 
parks once a year, not at all, or only when specific concerns are brought to their attention. 
We were informed that enforcement resources are allocated according to immediate 
priorities, which mainly involve the patrolling of operating parks. Most enforcement activity 
in non-operating parks is reactive and results from information received from volunteers, 
native groups, local police, park visitors, and other concerned citizens. 


Park superintendents also informed us that, because enforcement activity was inadequate in 
non-operating parks, they were not fully meeting their protection responsibilities. ‘The 
Ministry’s enforcement policies deal primarily with activities in operating parks, with no 
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specific guidelines outlining park wardens’ responsibilities in non-operating parks. 
Superintendents noted examples of illegal camping, hunting, cutting down of trees for 
firewood, use of recreational vehicles in parks, vandalism, and boundary encroachment by 
adjacent landowners. Some of these violations may be eliminated or reduced with periodic, 
risk-based patrolling, formalized in the Ministry’s minimum operating standards. 


Park superintendents noted that, as a result of the lack of enforcement, resources had been 
adversely affected and in some cases destroyed. However, they indicated that the extent of 
damage or effect on wildlife habitats cannot be determined, both because an inventory of 


what species or resources exist in the park is not available and because no statistics are 


maintained on the known damage to park resources. Therefore, there is a significant risk 
that the Ministry may not be maintaining provincial parks for the benefit of future 
generations as required by the Provincial Parks Act. 





Recommendation 


To help ensure that provincial park resources are adequately protected, the 
Ministry should: 


e review the level of enforcement activity in both operating and non- 
operating parks to determine whether there are adequate levels of funding, 
staff, and equipment for park superintendents and wardens to carry out 
their enforcement responsibilities; and 


e develop specific guidelines outlining a risk-based enforcement strategy for 
non-operating parks. 


Ministry Response 


The Ministry will undertake a review of the current park enforcement operating 
standard contained in the minimum park operating standards manual for 
operating parks. 


Since 1983 more than 150 new provincial parks (3.9 million hectares) have been 
added to the system. This significant expansion of the system has placed 
additional demands on enforcement activity in both operating and non- 
operating parks. Since 1996, the Ministry has increased its efforts in the area of 
park enforcement patrols. In 1996 there were 14,759 person-days of effort in 
enforcement, while in 2001 there were 18,987 person-days, representing a 28% 
increase. 





The Ministry will also develop a strategy for enforcement at non-operating 
parks based on the level of risk. 


bee 
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MEASURING AND REPORTING ON PROGRAM 
EFFECTIVENESS 


Measures of Ecological Sustainability 


In 1991, the Ministry produced a policy document termed Directions ’90s. That document 
indicated that the basis of future policy development would be the sustainable development 
of natural resources. Sustainable development is development that meets the economic, 
social, and environmental needs of the present generation without compromising the needs 
of future generations. Accordingly, throughout the 1990s, the Ministry attempted to 
develop desired outcomes to measure its success in achieving the sustainable development of 
natural resources. In 2000, the Ministry updated its policy with a document termed Beyond 
2000: Ministry of Natural Resources’ Strategic Directions. This document formally adopts the 
Ministry's objective of ecological sustainability and sets out the strategies that will help to 


achieve this outcome. 


VFM Section 3.07 





Ontario Parks’ 2000/01 Business Plan noted that the Program’s objective was to “protect 
and manage provincially significant natural, cultural and recreational environments in a 
system of provincial parks and provide a variety of outdoor recreational activities.” The 
Business Plan detailed a set of desired outcomes but did not define performance measures to 
assess the overall effectiveness of the Program in meeting its objective of ecological 


sustainability. 


Currently, the Program only collects information and reports on two performance measures. 
The first measure is the economic and social benefits of provincial parks expressed in terms 
of the number of park visitors. The second measure relates to the protection of the natural 
resource base with respect to endangered species in terms of the number of species 
regulated and number of recovery plans prepared and completed. ‘These measures relate to 
activities involved in operating provincial parks and do not indicate whether natural 
resources are being sustained. The Ministry has not developed effectiveness measures to 
specifically address the desired outcomes identified in the Business Plan and determine 
whether the Program is fulfilling the Ministry’s objective of ecological sustainability. 


Without an overall assessment of program effectiveness, such as the number of management 
plans prepared and implemented, the Ministry cannot determine whether its policies and 
management practices are successful in achieving its objective of ecological sustainability of 
provincial park resources. Ministry staff agreed that such an overall assessment was needed. 
However, information available was insufficient to measure the achievement of the 
Program’s objective. 

In this regard, in May 2001 the Ministry completed an ecological framework report 
entitled An Approach for Monitoring in Ontario’ Provincial Parks and Protected Areas. The 


purpose of this framework is “to assist Ontario Parks branch in the development of an 
approach to inventory, monitor, assess and report on long-term ecological health and 
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integrity, and the cumulative effect of human activities in Ontario’s provincial parks and 
conservation reserves.” Through this process, the Ministry hopes to identify indicator species 
to be monitored to determine the state of natural resources and ultimately whether there is 
ecological sustainability. The species and ecological changes are to be monitored and 
measured so that timely and effective action can subsequently be taken and ecological 
sustainability can be assessed. 


As a first step, in January 2002 the Ministry undertook a four-year assessment of its 
protected areas. The Ministry plans to assess pressures on the environment such as human 
recreational use, toxins and pollutants, habitat modifiers, climate changes, and variations in 
flora and fauna. The Ministry will use the results of the assessment as part of its monitoring 
of the ecological health and integrity of provincial park resources. 








Recommendation a 


The Ministry should develop performance measures for use in assessments 
that help to ensure the ecological sustainability of provincial park resources. 


Ministry Response 


The Ministry accepts the Provincial Auditor’s findings. With available 
resources, currently provided through short-term Ontario’s Living Legacy 
funding, the Ministry is taking several steps to address this concern including: 


¢ developing a Parks and Protected Areas Ecological Monitoring Program; 
and 


¢ completing year one of a four-year assessment of natural and human- 
induced stresses affecting Ontario’s provincial parks that, upon completion, 
will help the Ministry to assess the nature and degree of threats to the 
system and to set priorities on what indicators should be monitored and on 
how they should be monitored. 














DUE REGARD FOR ECONOMY AND EFFICIENCY 


Park Reservation and Accounting System 


In 1999, the Ministry entered into a five-year agreement with a private contractor to 
operate the Computer Reservation and Registration Accounting System, which is used by 
the public to make reservations for provincial park camping sites. The service provider 
operates a call centre and an Internet site for taking reservations. The contractor also 
provides the necessary personnel, as well as computer software and equipment. For the 
2000/01 fiscal year, the Ministry paid the provider approximately $2 million to operate the 


reservation system. 
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We reviewed the selection of the service provider and concluded that the vendor was 
competitively acquired in accordance with government policy. We also reviewed the 
performance of the service provider and the Ministry’s monitoring of security measures and 
procedures and had the following concerns: 


¢ The service provider has not met the customer service standard detailed in the 
agreement, whereby 80% of the telephone calls are to be answered within 20 seconds. 
The service provider is responsible for ensuring that adequate staffing levels are 
maintained to meet this standard. From February 2000 to September 2001, the 
Ministry determined that the standard was often not met and levied a penalty of 
$129,000 for the time periods when the service provider had not met the customer 
service standard. We conducted our own test and found that 65% of our calls were not 
answered either because of a busy signal or because we were put on hold for 15 minutes 
(after which time we hung up the telephone). At the completion of our audit in January 
2002, the service provider had not paid the penalty and, under the terms of the 
agreement, had asked for arbitration by a third party. 


¢ In February 2000, prior to implementation, the Ministry hired a consultant to test the 
system for security, and any problems noted were subsequently corrected. However, 
since February 2000 the Ministry has not carried out any other security tests of the 
system to verify that the security features to protect information being transmitted over 
the Internet were operating effectively. Management Board of Cabinet directives 
require that contracts with service providers include a provision for periodic 
independent security reviews of information technology facilities. The Ministry's 
agreement with the contractor did not include such a provision. 











Recommendation 


The Ministry should more closely monitor its service provider to ensure that 
customer service requirements are being met and ensure that future contracts 
with service providers include a provision for periodic independent security 
reviews. 





Ministry Response 


The Ministry monitors the performance of the reservation call centre on the 
basis of daily and weekly reports. The Ministry also monitors the quality of 
customer service through customer comments and complaints. On balance, 
the level of customer satisfaction is high, and the Ministry receives very few 
complaints regarding long hold times or busy signals. Nevertheless, the 
Ministry will undertake to enhance service-provider monitoring by dedicating 
resources to conduct random testing and sampling of response times. 


The Ministry will include a provision for periodic independent security review 
in future contracts with service providers. The Park Reservation and 
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Accounting System went on-line in March of 2000. The Ministry has now 
established a semi-annual security check of the Internet reservation system. 
The security checks are to be conducted by an independent consulting firm. 






Capital Infrastructure Maintenance 


The Ministry estimates that the value of the provincial park capital infrastructure, excluding 
land, is approximately $800 million. This infrastructure includes buildings, visitor centres, 
water distribution systems, sewage treatment facilities, roads, bridges, docks, and equipment. 
Since 1996, when Ontario Parks was created as a new business model, capital expenditures 
have been as outlined in the following table: 


Ontario Parks 
Capital Expenditures, 
1996/97—2001/02 


Capital Expenditure 
C) 


1996/97 9,579,000 
1997/98 9,801,000 


Fiscal Year 











1998/99 12,203,000 
1999/2000 13,781,000 
2000/01 19,303,000 
2001/02 15,569,000 














Source of data: Public Accounts of Ontario 


According to the Ministry, the majority of the existing capital infrastructure within the parks 
is between 20 and 45 years old and is approaching the end of or is beyond its useful life. 
Our review of the capital process indicated that the Ministry had a backlog of infrastructure 
projects and insufficient funding allocated to complete them. This results in the continued 
deterioration of the infrastructure facilities. Details of our concerns follow. 


Over 80% of the park superintendents responding to our survey indicated that funding for 
infrastructure maintenance was inadequate and did not fulfill the needs of the parks. They 
rated over two-thirds of the infrastructure facilities as marginal or unsatisfactory with 
required improvements outstanding for more than two years. The majority of these parks 
required improvements relating to washroom facilities, roads, playgrounds, water systems, 
and administrative buildings. In addition, customer comment cards indicated that 
satisfaction with park facilities decreased from 54% in 1997 to 45% in 2001. 


The Ministry estimates that the annual costs to maintain and rehabilitate its existing park 
infrastructure should be approximately $16 million. While annual capital funding that the 
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Program receives approximates this amount, part of the capital funding is designated for 
non-maintenance expenditures such as land acquisitions and new initiatives. For the 
2001/02 fiscal year, the Ministry estimated that an additional $8.5 million in capital 
funding was needed to adequately maintain the parks’ infrastructure. Consequently, 
deferring annual maintenance may result in accelerated structural deterioration that would 
eventually require more costly replacement. The Ministry estimated that the funding 
required to bring the capital infrastructure to an acceptable state was $420 million. 


Over 75% of the superintendents indicated that outstanding infrastructure deficiencies 
posed a possible threat to health and safety. For example, some staff housing did not meet 
the fire code, had structural problems, or contained mold that could cause health problems. 


In addition, by regulation under the Ontario Water Resources Act, park water treatment and 
distribution systems are required to meet provincial standards by December 31, 2002. 
Although the Ministry implemented a water improvement program in September 2000 to 
address water system deficiencies in provincial parks and comply with the requirements of 
the Ontario Water Resources Act, there is a risk that the Ministry will not meet the December 
31, 2002 deadline (subsequent to our audit, this deadline was extended to July 1, 2003). 
The Ministry had identified 58 projects that needed to be completed to comply with the 
requirements. However, as of January 31, 2002, most of the designs for rehabilitation were 
still in progress, with only one project having been approved by the Ministry of the 
Environment. In addition to the risk of missing the deadline, there is a risk that the Ministry 
will operate provincial parks for the 2003 season with water treatment and distribution 
systems that do not meet provincial standards. 











Recommendation 


To ensure that provincial parks are maintained for the benefit of future 
generations and to correct infrastructure deficiencies that may pose a threat to 
health and safety, the Ministry should take action to bring the parks’ 
infrastructure to a satisfactory state. 


Ministry Response 


The health and safety of park staff and visitors are of paramount importance to 
the Ministry. The Ministry has taken steps to ensure that any infrastructure 
deficiencies that may pose a threat to the health and safety of staff and visitors 
will be corrected. The Ministry will continue its ongoing efforts to restore the 
parks’ infrastructure with available resources. 





With the introduction of new drinking water standards, the Ministry has 
undertaken a major initiative to upgrade drinking water systems in provincial 
parks. The provision of drinking water in provincial parks meets current 
Ministry of Environment standards. The filtration requirements under the new 
regulatory standards of the Ontario Water Resources Act will come into effect 
on December 31, 2002. The Ministry will ensure that the provision of drinking 
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water in provincial parks for the 2003 season will meet the new provincial 
standards. To date, the Ministry has implemented the operational requirements 
of the new drinking water standards that came into effect in August 2000. 








Provincial Park Movable Assets 


We reviewed the Ministry’s management of provincial park movable assets such as furniture, 
tools, equipment, computers, and machinery. We concluded that the controls in place to 
properly account for and safeguard these assets were inadequate. Proper controls are 
essential as movable assets are susceptible to loss or theft. The Ministry had used an asset 
management system to record and track movable assets, but the system has not been 
operational since 1998, when incompatible technology was implemented. The Ministry did 
not replace the asset management system. 


Ministry policy requires that park superintendents maintain an up-to-date inventory list of 
the movable assets in their parks that is to be submitted to head office at least annually. 
However, many of the parks we visited had not maintained an inventory list since 1998 
when the asset management system was discontinued. The purchases from April 1, 1998 to 
December 31, 2001 of assets that should have been controlled and included on the asset list 
totalled $4.2 million. In addition, we found that the asset control function at these parks 
was a low priority for superintendents. Over two-thirds of the superintendents responding 
to our survey indicated that their asset inventory was not current and most dated back to 
1998. The Ministry also did not ensure that superintendents performed a periodic physical 
verification of assets. Without a complete listing of movable assets, the Ministry was not in a 
position to effectively monitor park assets. 


In January 2002, the Ministry released a new policy and guideline for movable assets 
detailing the mandatory minimum requirements for the effective management of movable 
assets. All assets are to be recorded and tracked using a new asset management system. 











Recommendation | 


To properly control and safeguard provincial park movable assets, the Ministry 
should develop and implement a new asset management system to permit the 
effective implementation of the new movable asset management policy and 
guideline. 


Ministry Response 


The Ministry agrees with the recommendation of the Provincial Auditor. The 
Ministry has implemented a new asset management system for the program 
areas that were using the older Asset Management Inventory system. In 
addition, park superintendents are required to conduct periodic verification of 
assets as per ministry policy and procedure. + 
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Ontario Parks Special Purpose Account 
Effective April 1, 1996 the Ministry established the Ontario Parks Special Purpose Account 


in the province’s Consolidated Revenue Fund. All revenues received under the Provincial 
Parks Act are to be deposited into this account and used, as directed by the Minister of 
Natural Resources, to make payments related to the care, preservation, improvement, 
control, and management of provincial parks. Park revenue is derived from user fees, 
equipment rentals, concession sales, and fines. During the 2000/01 fiscal year, revenue 


totalling $37 million was deposited to the credit of the Special Purpose Account. 


In order to properly account for all provincial revenue, the Financial Administration Act 
requires that all public money be deposited to the credit of the Minister of Finance and 
reflected in the Consolidated Revenue Fund. For some provincial parks, the Ministry has 
entered into agreements with third-party contractors to either operate the entire park or 
operate access points into the park. The Ministry has allowed contractors to retain all or a 
portion of the fees collected as park revenues and expenditures are understated in the Public 
Accounts of Ontario by the amount of revenue retained by contractors. 


We also reviewed revenue collection for ministry-operated parks. Daily revenue collection 
summaries are to be submitted at least weekly to the Ministry for reconciliation with deposits 
into the Consolidated Revenue Fund. Once the reconciliation is complete, the Ministry of 
Finance is to transfer the funds to the Ontario Parks Special Purpose Account. However, 
there have been significant delays in the Ministry’s reconciliation process that have 
prevented the timely transfer of revenue from the Consolidated Revenue Fund to the 
Special Purpose Account. Consequently, we estimate that the Ontario Parks Program has 
lost interest revenue that amounted to approximately $250,000 for the period from August 
1999 to October 2001. 


Recommendation 


To ensure that all public money is properly accounted for and the Ontario 
Parks Special Purpose Account earns all the interest it is entitled to, the 
Ministry should: 


¢ require that contractors deposit all provincial park revenue into the 
Consolidated Revenue Fund as stipulated by the Provincial Parks Act and 
the Financial Administration Act, and 


e perform the necessary reconciliations on a timely basis. 








Ministry Response 


The Ministry will undertake a review of the practice of allowing third-party 
contractors to retain all or a portion of the fees collected as reimbursement for 
their services to ensure compliance with applicable legislation. The Ministry is 
also taking steps to ensure that reconciliations are completed in a more timely 
manner. For the 2002 operating season, the Ministry has directed additional 
resources to revenue reconciliation. 
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MINISTRY OF PUBLIC SAFETY AND SECURITY 


3.08—Community Services 
Program 





BACKGROUND 


The Ministry’s Community Services Program (Program) is responsible for supervising all 
adult offenders (18 years of age and older) and young offenders (16 to 17 years of age) who 
are under some form of conditional release—that is, who are on probation, serving a 
conditional sentence, or on parole. 


Types of Conditional Release and Relative Percentages 


ee Percentage of 
_ Typeof Total Offenders 
_ Conditional Description Being 
Release _ Supervised in 
_ _ the Community 


Probation Probation is a sentence imposed by the courts that allows 
order offenders to serve their sentence under supervision in the 
community. 


Probation is generally focused on rehabilitation. 














Conditional A conditional sentence is imposed by the courts and allows 
sentence offenders to serve their sentences under supervision in the 
community. The option of conditional sentences was 
introduced in 1996. 
Conditional sentences are generally intended to be both 
punitive (through conditions like house arrest and curfews) 
and rehabilitative. 








Parole Parole is the early release of an offender from a correctional 

certificate institution under supervision in the community. Paroles can 
only be authorized by the Ontario Parole and Earned Release 
Board to eligible inmates after they have served one-third of 
their sentences in correctional institutions. 





Prepared by the Office of the Provincial Auditor of Ontario 
Source of data: Ministry of Public Safety and Security 


The objectives of the Program are to protect the public by monitoring offenders in the 
community and to rehabilitate offenders through training, treatment, and services that 
afford them the opportunities for successful personal and social adjustment in the 
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community. The Ministry’s mandate in this respect and provisions for probation and parole 


are defined in the Ministry of Correctional Services Act. 


On any given day, there is an average of 65,000 offenders being supervised by the Ministry 
in the community. This represents almost 90% of Ontario’s total correctional population. 
The other 10% consists of offenders who are in institutions serving sentences or on remand 
awaiting trial. 


At the time of our audit, the Ministry employed approximately 770 probation and parole 
officers working in 41 area offices and 86 sub-offices throughout the province. In addition, 
as part of its Community Services Program, the Ministry also contracts with selected 
community agencies to provide a variety of non-residential services, such as substance abuse 
treatment, psychological therapy, and other counselling/treatment programs. 


In 2001/02, total program expenditures amounted to $82 million—$63 million of which 
was spent on salaries and benefits. 


AUDIT OBJECTIVES AND SCOPE 


The objectives of our audit were to assess whether adequate procedures were in place to 
ensure that: 


¢ offenders serving sentences in the community comply with the conditions of probation, 
conditional sentences, and parole; 


e the Ministry measures and reports on the effectiveness of its services and programs; and 
ry p prog 


¢ the Ministry manages its human and financial resources with due regard for economy 


and efficiency. 


Prior to the commencement of our audit, we identified the criteria that would be used to 
conclude on our audit objectives. These were reviewed and accepted by senior ministry 
management. Our audit was conducted in accordance with standards for assurance 
engagements, encompassing value for money and compliance, established by the Canadian 
Institute of Chartered Accountants, and accordingly included such tests and other 
procedures as we considered necessary in the circumstances. 


The scope of our audit, which was substantially completed in March 2002, included visits 
to five area offices, interviews with ministry officials, examinations of contracts with 
community service agencies, and reviews of offender files. We did not rely on the Ministry's 
internal auditors to reduce the extent of our work because they had not recently conducted 
work within the scope of our audit. 
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OVERALL AUDIT CONCLUSIONS 


While acknowledging that the Ministry is in the process of implementing a new risk-based 
offender management model, we concluded that there were still a number of deficiencies in 
its procedures that hindered the effective supervision of offenders in the community. 


Specifically: 


The Ministry had not completed the required risk and needs assessments for many 
offenders under its supervision to reduce the risk that these individuals would reoffend 
while in the community and to ensure rehabilitation needs were identified. At the 
offices we visited, over 40% of offenders who had committed additional “level I” 
offences while under ministry supervision lacked a risk and needs assessment and/or a 
management plan. (Level I offences include sexual assault, assault causing bodily harm, 
uttering death threats, and other violent crimes.) 


When offenders did not comply with supervision conditions, the Ministry often did not 
take corrective action on a timely basis. At the five offices we visited, of the cases 
involving level I offenders who later committed additional offences while under ministry 
supervision, we noted that over 30% had not been followed up on a timely basis after 
the offender failed to comply with the conditions of their supervision. 


We estimated there were approximately 10,000 arrest warrants outstanding for 
offenders in the community, and warrants for some of these offenders had been issued 
as far back as 10 years. (Arrest warrants are issued when offenders fail to report to their 
probation and parole officer and cannot be located.) Many of these offenders were 
assessed as high risk and had committed level I offences. The Ministry did not know 
how many of the offenders against whom there were arrest warrants outstanding were 
still at large. 


While we recognize that once a warrant is issued, the police—not ministry staff—are 
responsible for apprehending the offenders, the Ministry and the police need to work 
more closely together to capture these high-risk offenders so as not to expose the 
community to significant risk. 


The Ministry had not adequately addressed the correctional needs of offenders in the 
community. According to internal ministry reports, correctional programs for offenders 
with addiction problems, anti-social behaviour, personality disorder, and sexual deviance 
were often not available in their local communities. For example, of the over 3,000 sex 
offenders being supervised by the Ministry in the community, less than 600 received 
appropriate rehabilitation programs. 


Probation and parole officers indicated to us that current caseload levels impaired their 
ability to effectively supervise and provide services to offenders in the community. In 
spring 2001, the Ministry was authorized to hire 165 more probation and parole 
officers (it then had 690) to address identified workload concerns. Although the 
Ministry had previously advised us that workload standards to determine the optimal 
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number of officers needed for efficient and effective delivery of services would be in 


place by 1996, these standards have still not been developed. 


DETAILED AUDIT OBSERVATIONS 
NEW OFFENDER MANAGEMENT MODEL 


For many years, the Ministry has had in place an empirically based risk-assessment process 
that its probation and parole officers use to assess the risk that an offender will reoffend 
while under ministry supervision in the community. However, there was no process in place 
to deploy staff resources based on the results of these assessments. 


We were pleased to note that since our 1995 audit of the Program, the Ministry, in 1999, 
initiated a new offender management model known as the Probation and Parole Service 
Delivery Model, which highlights offenders’ correctional needs that should be addressed to 
effectively reduce the risk of offenders reoffending. Though not yet fully implemented, the 
new model is expected to concentrate resources on those offenders at a high risk of 
reoffending. Under the model, empirically based approaches in offender assessment and 
supervision, as well as rehabilitation programs, which have been shown to have positive 
impacts on reducing reoffending rates, are being incorporated into the day-to-day practices 
of probation and parole officers. 


Based on assessed risk and needs, the new model places offenders in one of four intervention 
service streams to meet their correctional needs. The most intensive level of supervision is 
reserved for offenders assessed as being at greatest risk of reoffending. According to the 
Ministry, over 20% of offenders in Ontario are considered to be in this high-risk category, 
and another 30% are considered to be at medium risk. Effective implementation of the new 
model is expected to result in more efficient deployment of limited resources and a 
reduction in reoffending rates. 


At the time of our audit, we noted that about 110 (over 80%) of the 127 probation and 
parole offices were in the process of implementing the new model. Of the 110, 39 had 
achieved full implementation. Ministry management has indicated that such major change 
in service delivery would take three to five years to fully implement. 


We noted that the Ministry has already begun an evaluation process to assess whether offices 
where the model has been implemented are complying with the new model. We will follow 
up in two years on the progress of the implementation of the model. 


I 
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SUPERVISING OFFENDERS IN THE COMMUNITY 


Risk and Needs Assessments and Management 
Plans 


Regardless of the offender management model being used, effective supervision of offenders 
is crucial to the prevention of further reoffending. In this respect, probation and parole 
officers are required to complete a thorough risk and needs assessment and an 
individualized management plan to address the risks and needs for all offenders who report 
to them. According to ministry policy, probation and parole officers must complete each 
offender's risk and needs assessment and management plan within six weeks of the date of 
initial contact with the offender. 


In the majority of offender files that we examined, we found that these key requirements of 
supervision—assessing risk and correctional needs and completing management plans— 
were not being met. These deficiencies existed even in the files of offenders who had 
committed level I offences, including sexual assault, assault with a weapon, and other violent 
crimes. At the offices we visited, over 40% of the offenders who had committed additional 
level I offences while under ministry supervision lacked a risk and needs assessment and/or a 
management plan. 


Given the importance of risk assessments and management plans to the effective supervision 
and rehabilitation of offenders, not meeting these key requirements places the safety of the 
community at undue risk and does not assist offenders in adjusting successfully in the 
community. 








Recommendation 





To reduce the risk that offenders under ministry supervision will reoffend and 
to enhance the rehabilitation of these offenders, the Ministry should complete 
the required risk and needs assessments and management plans for these 
offenders on a timely basis. 


Ministry Response 


The Ministry is committed in policy and practice to timely completion of 
assessments and case plans for offenders under its jurisdiction. 


The Ministry is addressing compliance with this policy in a number of ways. 
The hiring of the additional 165 probation and parole officers will facilitate 
workload distribution to improve more timely completion of offender 
assessments and management plans. In addition, the Ministry is reviewing 
current policy in an attempt to streamline the assessment requirements to 
ensure prioritizing and focusing of resources on higher-risk offenders. The 





Ministry is also finalizing revisions to probation and parole caseload audit 
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procedures to improve accountability, quality assurance, and performance 
management. 


The initial impact of learning a new technology—that is, the Offender Tracking 
Information System (OTIS)—has affected the timelines of assessment and 
management plans input into the electronic system. It is expected that full 
implementation of this new business process will ultimately contribute to more 
accurate and timely access to information about offenders. 





Through the collaborative evaluation process of the Probation and Parole 
Service Delivery Model, congruence with service delivery policy and practices 
is being evaluated. Based on evaluation findings, parole offices are required to 
put plans in place to improve in areas of deficiency. 














Monitoring and Enforcing Compliance with 
Conditions of Supervision 


A primary role of probation and parole officers in protecting public safety is to ensure 
offenders comply with the conditions outlined in supervision documents (probation orders, 
conditional sentences, and parole certificates). The type of conditions that offenders must 
comply with include: reporting to a probation and parole officer, performing community 
work, refraining from associating with any person engaged in criminal activity, attending 
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treatment or counselling programs. With respect to probation and parole officers ensuring 
compliance with conditions, we found weaknesses in following up when offenders failed to 
comply with supervision conditions, even in cases of high-risk offenders. For instance, at the 
five offices we visited, of the cases involving level I offenders who later committed additional 
offences while under supervision, we noted that over 30% had not been followed up ona 
timely basis after the offender failed to comply with the conditions of their supervision. 


In each instance of non-compliance, a probation and parole officer is required to make an 
enforcement decision and take appropriate action. Enforcement decisions can range from 
taking no action, to verbal or written cautions, to increased supervision, to seeking from the 
court an amendment to the supervision conditions, to charging the offender with a breach 
of conditions. In cases of probation and conditional sentences, the decision to initiate a 
charge of breach is at the discretion of the officer. In the case of parole, all violations must be 
reported to the Ontario Parole and Earned Release Board. In all cases, probation and parole 
officers are required to document the nature of the violation, the enforcement decision 
made, and the rationale for the decision. 


Our audit revealed that in cases where offenders did not comply with supervision orders 
and enforcement action was not taken to address this non-compliance, almost two-thirds of 
the files we reviewed did not provide explanations as to why action was not taken. 


In addition, with respect to conditional sentences involving house arrest or curfews as 
required by the courts, probation and parole officers informed us that they were unable to 
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ensure that these offenders were adhering to the conditions of their sentences because they 
have no means of monitoring compliance with such conditions. The Ministry indicated it 
was in the process of issuing a request for proposals under its electronic surveillance program 
to identify the best available technology and devices that it can utilize to ensure conditions 
for house arrests and curfews can be monitored. 








Recommendation 


To better ensure public safety, the Ministry should: 


e take timely and appropriate corrective action when offenders under 
ministry supervision fail to meet the conditions of their supervision, 
especially in cases of high-risk offenders; and 


¢ ensure probation and parole officers properly document their decisions, 
including the rationale for not taking enforcement action in cases of non- 
compliance. 


Ministry Response 


The Ministry is committed to ensuring that its mandate of monitoring and 
enforcing conditions is met. 


Ministry policy emphasizes the need for timely enforcement decisions and 
documentation in cases of non-compliance. In all cases where non-compliance 
occurs and enforcement action is not taken, the rationale for not doing so is to 
be thoroughly documented as an enforcement decision. Probation and parole 
officers will be reminded of the requirement to document all enforcement 
decisions. 


Workload issues have been noted as affecting probation and parole officers’ 
ability to complete all duties in a timely manner. The Ministry is continuing to 
review ways of reducing workload for probation and parole officers. As well, 
the Ministry has hired an additional 165 probation and parole officers in order 
to provide more intensive supervision and monitoring of higher-risk offenders, 
thus ensuring public safety. 








Outstanding Arrest Warrants 


Arrest warrants are issued by courts at the request of probation and parole officers when 
offenders have failed to report to their probation and parole officers and cannot be located. 
Such offenders pose a risk to the community because the Ministry is unable to supervise 
them, and they may reoffend while at large in the community. 


In the five area offices we visited, we noted there were over 1,500 offenders against whom 
there were arrest warrants outstanding, and some of the related warrants dated as far back as 
10 years. Our review found that over 30% of these arrest warrants were for offenders who 
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were assessed to be high risk and had committed level I offences (including sexual assault, 
assault causing bodily harm, uttering death threats, and other violent crimes). We estimated 
that province-wide there could be over 10,000 outstanding arrest warrants. The Ministry 
did not have records indicating how many offenders against whom there were arrest 
warrants outstanding have not been apprehended. 


We recognize that once a warrant is issued, the police—not ministry staff—are responsible 
for apprehending the offenders. However, the Ministry and the police need to work more 
closely together to capture these high-risk offenders so as not to expose the community to 
significant risk. 





es . 
Recommendation | 


To better protect the safety of the community and enhance the credibility of the 
justice system, the Ministry should work more closely with the police to ensure 
that high-risk offenders against whom there are arrest warrants outstanding 
are apprehended in a timely manner. 


Ministry Response 


As noted by the Provincial Auditor, the police are responsible for 
apprehending offenders with outstanding arrest warrants. The Ministry 
recognizes the importance of apprehending high-risk offenders against whom 
there are arrest warrants outstanding, and Correctional Services is working 
with the Policing Services Division of the Ministry of Public Safety and Security 
to address this issue. 


In addition, the Ministry has already demonstrated its commitment to working 
closely with the police and will build on the following initiatives to ensure that 
high-risk offenders against whom there are arrest warrants outstanding are 
apprehended in a timely manner: 


e As acomponent of the Probation and Parole Service Delivery Model, to 
improve intensive supervision for offenders who are at a higher risk to 
reoffend, strong linkages between probation and policing services have 
been developed. Each probation and parole office is directed to work with 
local police agencies to develop protocols for the enhanced management of 
offenders who are assessed as posing a higher risk of reoffending. These 
protocols will be reviewed in the context of this recommendation. 


e In August 2001, the government approved the implementation of a 
provincial enforcement unit, the Repeat Offender Parole Enforcement Unit, 
which focuses on the apprehension of parole violators/fugitives as well as 
persons identified as being unlawfully at large throughout the province. 
Correctional Services will work with this unit and other regional and 
municipal police forces to explore solutions to ensure the apprehension of 
high-risk offenders against whom there are arrest warrants outstanding. 
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As well, the Integrated Justice Project, when fully implemented, will permit — 
offender and offence information to be shared across the justice sectors, 
including the Canadian Police Information Centre. Integration of the different 
sources of information will lead to greater and more timely access to _ 
information and a justice system that is more accessible, efficient, and 
effective. 







REHABILITATION PROGRAMS 


While risk assessments determine the likelihood of reoffending, rehabilitation programs are 


aimed at addressing the correctional needs of offenders in the community and thereby 
reducing the risk of reoffending. Research done by the Ministry shows that punishment 
and surveillance are not effective in reducing reoffending rates. Instead, what is effective in 
reducing reoffending rates is addressing the correctional needs of offenders through 
rehabilitation programs, such as anger management and substance abuse treatment. 


On average, offenders spend over one year under supervision in the community, and 70% 
of offenders spend more than six months under supervision. Such lengthy supervision 
periods give the Ministry an opportunity to provide rehabilitation programming that is 
geared to reducing the risk to the community by addressing the correctional needs of these 
offenders. 


The Ministry has recognized the need to provide rehabilitation treatment and programs to 
offenders in the community. In fact, we noted at the time of our audit that the Ministry had 
developed three core rehabilitation programs designed to address the correctional factors 
that have been empirically determined to be the most common within the offender 
population and had implemented these programs in 39 of its 127 probation and parole 
offices. These programs are aimed at anger management, substance abuse treatment, and 


anti-criminal thinking. 


Nevertheless, the Ministry’s current Community Services Program focuses more on 
supervisory conditions than on rehabilitation services. Supervisory conditions include 
measures like community service orders, which are court-ordered sanctions that require 
offenders to perform a specified amount of unpaid work in the community under the 
supervision of a probation and parole officer or contracted agency. In 2000/01, of the 
$8 million the Ministry spent on community contracts, about $2 million was spent on 
rehabilitation programs while about $6 million was spent on supervisory conditions. 


According to an internal ministry report, correctional programs to meet the needs of 
offenders with addiction problems, anti-social behaviour, personality disorder, and sexual 
deviance were generally not available. For example, the report indicated that of the over 
3,000 sex offenders being supervised by the Ministry in the community, fewer than 600 
received appropriate programs. This lack of rehabilitation programs to meet the 
correctional needs of offenders diminishes the effectiveness of community supervision. 
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Recommendation 


To provide offenders under the Ministry’s supervision with better opportunities 
for successful personal and social adjustment in the community, the Ministry 
should ensure the availability of rehabilitation programs that offenders need. 


Ministry Response 


The Ministry is committed to promoting the increased availability of core 
programs throughout the correctional system through the combined efforts of 
the Core Program Consultant (a newly dedicated position), the Program 
Effectiveness Unit, and staff working under the Probation and Parole Service 
Delivery Model. 


Under the Probation and Parole Service Delivery Model, core programs 
address substance abuse, anger management, and anti-criminal thinking. As 
well, specialized programs for partner abuse and sexual offenders are offered. 
There are two levels of core rehabilitative programs: a general orientation 
rehabilitative group and an intensive rehabilitative group. Important 
components of the core rehabilitative programs are the achievement of 
consistency in approach across the Ministry and the continuity of 
programming between the institutional part of the Ministry and Probation and 
Parole. Assessment and evaluation are key components to determine the 
effectiveness of the intervention in reducing offender recidivism. 


A concerted effort is underway to expand the availability of core programs to 
all Probation and Parole offices. _ 








PROBATION AND PAROLE OFFICERS 


Caseloads and Workloads 


The number of cases that probation and parole officers supervise has increased almost 8% 
in five yearr—from 60,000 offenders under supervision in 1995/96 to 65,000 offenders in 
2000/01. This increase in the caseload is due mainly to the introduction of conditional 
sentences as a sentencing option in 1996. 


Prior to spring 2001, the Ministry had 690 probation and parole officers. Recognizing the 
need for more resources to handle the increase in caseload, the Ministry was authorized to 
hire, in two phases, an additional 165 officers. This would represent about a 20% increase 
in staff, The Ministry estimated that this will eventually lower the average caseload per 
officer to about 85. At the time of our audit, the average caseload per officer was 95. The 
average caseload per officer for other Canadian provinces was about 70, but according to 
the Ministry it would not be fair to make a comparison with other jurisdictions because of 
differences that may exist in the responsibilities of officers, including administrative 
responsibilities. 
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Probation and parole officers’ work involves supervision, enforcement, and providing 
counselling and referral services to offenders in accordance with the offender’s assessed risks 
and needs. Probation and parole officers are also required to prepare court reports, attend 
court for trials, confirm collateral information (such as an offender’s address and 
employment), and liaise with community service agencies. In addition to their caseloads, 
with the ongoing implementation of the new Probation and Parole Service Delivery Model, 
many probation and parole offers are also delivering rehabilitation programs such as anti- 
criminal thinking, substance abuse treatment, and anger management. 


Moreover, with the introduction of conditional sentencing in 1996, the profile of offenders 
being supervised in the community has changed. Many offenders who would have 
previously been incarcerated are now serving their sentences in the community. These 
offenders include those with mental health, substance abuse, and other problems. This 
change in profile has resulted in the need for officers to increase their counselling and 
rehabilitative efforts to contribute to the safety of the community. 


Our discussions with probation and parole officers indicated that a greater number of cases 
and increased responsibilities have impaired the ability of officers to do their jobs effectively. 
While the current supervision model expects probation and parole officers to be actively 
involved with community agencies and with offenders in the community, officers describe 
their jobs as being “office bound” and indicate that larger caseloads impair their ability to 
conduct home visits and to liaise with community partners engaged in prevention or 
collaborative initiatives designed to meet offenders’ identified needs. 


In our previous audit of this program in 1995, we noted that the Ministry recognized that 
workload was a concern to probation and parole officers and area office management and 
had been since 1989. In 1995, we recommended that the Ministry establish workload 
standards and use them to analyze staffing so that staffing could be deployed in a more 
efficient manner. At that time, the Ministry indicated that a probation/parole workload 
index would be implemented by February 1996. When we followed up in 1997, the 
implementation of the index had been delayed for later in 1997. During this audit, there 
was still no workload index in place. 


Ministry management indicated that, because of the implementation of the new Probation 
and Parole Service Delivery Model and the introduction of a new Offender Tracking and 
Information System (OTIS), the workload index that was being developed in 1997 was no 
longer applicable. For instance, under the new service delivery model, officers are now 
leading rehabilitation groups and are engaged in other activities that were not reflected in 
the range of assignable activities when the workload index was developed. 


In the absence of a workload index, managers lack the appropriate tools to balance officer 
caseloads and workloads within their area offices. Similarly, without appropriate workload 
standards, the Ministry cannot determine the optimal number of officers needed and 
cannot effectively allocate work and deploy staff among area offices throughout the 
province. 
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Recommendation 


The Ministry should develop workload standards and use them to analyze 
staffing requirements so that staff can be deployed in a more efficient and 
effective manner. 








Ministry Response 


In 1997, the Ministry developed and tested a workload index to assist in the 
measurement of probation and parole workload. As a result of the difficulties 
associated with the weighting of various workload factors (including 
geographical disparities), this tool was not widely used. It should be noted that 
the complexities of developing a workload index have also been experienced in 
other jurisdictions. Furthermore, the 1997 workload index is outdated as it 
does not account for the changes resulting from the Probation and Parole 
Service Delivery Model or the Offender Tracking Information System. 





Although some workload factors will continue to be very difficult to measure 
(for example, delivering programs in a group setting), the Ministry is committed 
to developing a framework for workload measurement within the context of its 
new service delivery model in order to effectively allocate staff and distribute 
workload. 





OFFENDER TRACKING AND INFORMATION 
SYSTEM 


As part of a larger modernization project of the justice information system—the Integrated 
Justice Project—the Ministry has implemented a new electronic Offender Tracking and 
Information System (OTIS) to replace the former Offender Management System. OTIS is 
an Internet-based system that allows for the sharing of offenders’ information among 
partners in the justice system, including police, Crown attorneys, and the courts. Our 
review and discussion with ministry staff indicated that while OTIS supports the sharing of 
information with other partners, it does not facilitate probation and parole officers’ case 
management. Specifically: 


¢ The design of the new system does not allow officers to easily and readily assess an 
offender’s history and the types of offences committed (because the information is not 
captured and displayed on one screen but is instead stored and arranged among several 
different screens). As a result, there is an increased risk that critical information is missed 
when officers are reviewing information concerning offenders under their supervision. 


¢ The Internet-based system is vulnerable to frequent crashes. As a result, case 
management notes can be lost without warning. 


In addition, in all the offices we visited, probation and parole officers complained that 
excessive time was being spent on the case management component of the system. 
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Previously, case management notes were captured using a simple word-processing 
application, which enabled probation and parole officers to easily record and exchange 
offender case notes internally. Officers also indicated that case management notes contain 
confidential information not to be shared with other partners, and it was therefore not 
necessary to have such notes included in the Internet-based system. 


In our 2001 Annual Report, we also noted that there were inadequate controls to prevent 
unauthorized access to offender records. Ministry senior management indicated that it is in 
the process of making improvements to the system. As an interim measure, offices have been 
given the option to use the previous word-processing application for their case management. 


We recognize that the system is new and that the Ministry is taking measures to address the 
problems identified. We will follow up in two years on the Ministry’s progress towards 
correcting the problems with its information system. 


FUNDING AND MONITORING COMMUNITY 
SERVICE AGENCIES 


The Ministry contracts with selected community agencies to provide a variety of non- 
residential programs and services, including substance abuse programs, psychological 
therapy, counseling/treatment programs, and the supervision of community service orders 
(court ordered sanctions that require offenders to perform unpaid work in the community). 
In some cases, these programs are prescribed on an individual basis by the court or the 
Ontario Parole and Earned Release Board. In other cases, the probation and parole officer 
makes the appropriate referral after completing a comprehensive assessment of the 
corrective needs of the offender. 


In 2001/02, funding to these community service agencies amounted to $8 million. This 
funding is based on annual budgets that agencies submit. 


While the Ministry is not involved in the day-to-day operations of these agencies, ministry 
policy requires that contracted agencies comply with ministry standards and guidelines for 
supervision, security, and services. At the time the contract is negotiated, clear expectations 
of performance and documentation are to be established with each agency—including a 
ministry review of caseload statistics, offender progress reports, and financial records—to 
ensure that expenditures are properly recorded and disbursed in accordance with ministry 
policies. 


Our review of arrangements with community service agencies with the five area offices we 
visited revealed the following: 


¢ Instead of funding agency programs based on a proper assessment of the service levels 
required to meet the needs of offenders, the Ministry was funding the programs based 
on historical costs. Management indicated that the historically based funding approach 


was a function of a lack of resources allocated to the programs. 
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¢ Contracts were generally not signed on a timely basis. In some cases, contracts were 


signed almost 11 months after the agency began providing services. In fact, for six of the 


12 contracts we examined, approximately $220,000 was paid to agencies before the 


required contracts were in place. Furthermore, under two of these six contracts, agencies 


received their full funding before their contract was in place. 


¢ Monitoring and feedback mechanisms established by the Ministry to ensure agency 
accountability—such as offender case statistics, progress reports, and financial 
statements—were not received in the majority of the cases we reviewed. 


Because the Ministry did not properly assess service-level requirements and did not 
adequately monitor the services being delivered by agencies, it could not assess whether 
funding levels were appropriate and whether agency programs were meeting the needs of 


the offenders. 





Recommendation 


To ensure both due regard for economy and efficiency and accountability for 
service performance, the Ministry should ensure that: 


e funding to community service agencies that provide programs to offenders 
is based on a proper assessment of service-level requirements; 


¢ payments made to these community service agencies are properly 
supported by signed contracts; and, 


e services provided by such agencies are monitored to confirm that they 
adhere to ministry standards and meet the needs of offenders and that 
funds are used prudently. 


Ministry Response 


The Ministry is committed to ensuring that services provided by community 
agencies are funded appropriately, are monitored to meet ministry standards, 
and meet offender needs. A template for a “letter of intent” was developed for 
use with contracted agencies to allow for the continuation of annualized 
funding until contract negotiations are complete. 


Draft performance outcomes—a measurement of various related indicators 
that define the performance expectations of the standards of operations—have 
been developed in the management of open-custody facilities. Following the 
completion of a consultation process with open-custody operators, the new 
performance-measurement framework will be implemented in the 2002/03 fiscal 
year. 


The Program Effectiveness Unit has submitted a proposal to Management 
Board Secretariat for funding to conduct a systematic evaluation of contracts, 
and a decision is pending. 
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MEASURING AND REPORTING ON PROGRAM 
EFFECTIVENESS 


The rate of recidivism is the most commonly used measure of the performance of the 
correctional system. While there is strong public and government interest in using 
recidivism rates to judge the performance of the correctional system—both in terms of 
effectiveness and efficiency—there is no generally accepted definition of recidivism in 
Canada or internationally. 


To its credit, the Ministry has defined recidivism and intends to measure its performance 
based on this definition. It defines recidivism as a return to correctional supervision 
following conviction for a criminal offence committed either during or after correctional 
supervision. 


However, we noted that the Ministry had not yet measured and reported on the 
effectiveness of community supervision and rehabilitation programs in contributing to 
public safety and the re-integration of offenders into the community. At the time of our 
audit, the Ministry was still in the process of establishing baselines and standards to measure 
and report on recidivism. 








Recommendation 


The Ministry should implement performance measures to assess the 
effectiveness of the Community Services Program in contributing to public 
safety and the rehabilitation of offenders. 


Ministry Response 





The Ministry introduced a new performance framework as part of its 
transformation strategy to apply to all adult correctional institutions and will 
apply a similar performance framework across community correctional 
supervision. The reduction of reoffending rates is a key focus of the 
performance framework. Work that has commenced regarding performance 
measures and indicators for community corrections includes measures 
relating to recidivism, enforcement activity, intensive supervision, training for 
probation and parole officers, and the per diem cost per offender for 
community supervision. 


The Ministry developed a recidivism definition—which then led to the 
establishment of baseline rates—using representative release samples from 
1992, 1993, and 1997. Since the implementation of the Offender Tracking 
Information System, work is underway to gather the necessary data in order to 
measure the effectiveness of correctional policies and programs related to the 
rehabilitation of offenders. The Ministry expects to gael data collection 





processes in 2002/03. 
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MINISTRY OF PUBLIC SAFETY AND SECURITY 


3.09—The Ontario Parole 
and Earned Release Board 





BACKGROUND 


The Ontario Parole and Earned Release Board (Board) derives its authority from the federal 
Corrections and Conditional Release Act and the provincial Ministry of Correctional Services 
Act. It makes decisions about parole for offenders sentenced to less than two years of 
imprisonment. Offenders are eligible for parole consideration upon serving one third of 
their sentences. Offenders who are granted parole serve the remainder of their sentences in 
the community under the supervision of a parole officer. Without parole, offenders are 
normally released after serving two thirds of their sentences. 


The National Parole Board makes parole decisions on behalf of all other provinces except 
British Columbia and Quebec. The National Parole Board also makes parole decisions for 
all offenders who have been sentenced to imprisonment for two or more years because those 


offenders fall under federal jurisdiction. 


The Correctional Accountability Act, proclaimed on July 3, 2001, changed the former 
Ontario Board of Parole into the current Board and expanded its traditional role of making 
decisions on parole to include decisions on whether offenders have earned enough credit 
days while incarcerated (by participating in rehabilitative programs, demonstrating positive 
behaviour, and being drug-free) for release. The Board was also given the responsibility for 


decisions about granting temporary absences over 72 hours. 


At the time of our audit, the Board had four full-time members, including the Chair, and 
over forty part-time members. Most full-time members have a correctional services or 
criminal justice background; part-time members are lay people representing their local 
communities. A quorum of two members is required to conduct a parole hearing, In case of 
a split decision, a new hearing is scheduled with three different members attending. 


For the 2001/02 fiscal year, the Board’s total expenditures amounted to approximately 
$3 million. For the 2000/01 fiscal year, approximately 2,100 parole hearings were held, 
28% of which resulted in parole being granted. 
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AUDIT OBJECTIVES AND SCOPE 


The objectives of our audit were to assess whether the Board had adequate procedures in 
place to ensure that parole decisions were made in accordance with legislative and board 
policies as well as to measure and report on its effectiveness in contributing to public safety 
and facilitating the reintegration of offenders into society. 


Prior to the commencement of our audit, we identified criteria that would be used to 
conclude on our audit objectives. These were reviewed and accepted by the Board’s Chair. 


Our audit was conducted in accordance with the standards for assurance engagements, 
encompassing value for money and compliance, established by the Canadian Institute of 
Chartered Accountants, and accordingly included such tests and other procedures as we 
considered necessary in the circumstances. 


Our audit work, which was substantially completed in March 2002, included visits to the 
Board’s head office and three of its four regional offices. It also included an examination of 
parole files and other internal board documents, discussions with board and ministry 
officials, observation of parole hearings, and research into parole practices in other 
Canadian jurisdictions. Our audit did not include the Board’s decisions on earned release 
and temporary absences over 72 hours, as the Board had not made any decision on earned 
release and implementation of the Temporary Absence Program began only in January 
2002. 


OVERALL AUDIT CONCLUSIONS 


We concluded that the Board’s mandate of protecting society by effectively reintegrating 
offenders into the community was hindered by a dramatic reduction in the number of 
eligible inmates being considered for parole. Specifically, the decline in the number of 
hearings from 6,600 to 2,100, combined with a steady drop in parole grant rates from 59% 
to 28%, has resulted in fewer than 600 inmates being granted parole in 2000/01, as 
compared to 3,800 in 1993/94. 


According to board studies, factors contributing to this decline included inmates not 
receiving the required parole information and inmates waiving parole hearings because they 
felt there was little chance of getting a fair and unbiased hearing. 


In one of the four regions, we found that it was a matter of practice to deny any applications 
for a parole hearing from inmates serving 122 days or less, thus depriving a significant 
number of offenders of the opportunity to have their cases heard. This is particularly 
significant in that 85% of Ontario’s inmates generally serve sentences of less than six months 
and on average are sentenced to only about 70 days. 
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We also concluded that: 


The Board had set performance goals for 2001/02 below those already achieved; thus, 
its goals do not serve to encourage an improvement in board performance. 


The Board had not conducted quality assurance and performance reviews as required 
by board policy. In cases where reviews were done, the results of the reviews were not 
used to improve the quality of decision-making and the performance of board 


members. 


Ontario had no formal selection process to assess the abilities, skills, commitment, and 
suitability of potential board members, nor did the Board have the opportunity to 
provide input on the initial screening of potential candidates. 


The Board had not recorded its rationale for its decisions not to impose special 
conditions that had been recommended by parole officers or police. Examples of 
recommended conditions included the requirement that parolees have no contact with 
their victims and the requirement that parolees do not possess firearms. 


In addition, we found that, although Ontario’s parole grant rates have significantly declined 


since 1993/94, its rates of parolees reoffending during parole have been generally higher 
since that same time. This situation requires research by the Board to determine what 
further action, if any, is required. 


It is worthy to note that, with regard to reviewing reports from parole officers, modifying 


parole conditions, and, when necessary, issuing revocations and arrest warrants, the Board 
acted in an appropriate and timely manner. 











Overall Board Response — 


The Ontario Parole and Earned Release Board and the Ministry of Public 
Safety and Security are in the midst of a comprehensive review of the Board's 
organizational and accountability structure. This review will significantly 
enhance the Board's ability to implement recommendations made by the 
Provincial Auditor. 


The review was prompted by the recent construction/enlargement of three 
provincial correctional centres and concurrent closure of many older, smaller 
facilities. These three institutions will soon house 72% of Ontario’s inmates 
serving sentences of six months or more. 


The rationalization of ministry correctional facilities will require the Board to 
restructure its resources. While the bulk of the Board’s work will take place at 
the three new facilities, the Board’s offices are currently spread out across the 
province. Ideally, these offices should be co-located in provincial correctional 
institutions. The Board is also reviewing options to reorganize its management 
structure to promote efficiencies and best practices and move from a paper- 
based to an electronic hearing system. 








I —————— 
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DETAILED AUDIT OBSERVATIONS 


The Corrections and Conditional Release Act states that the purpose of parole is to 
“contribute to the maintenance of a just, peaceful and safe society by means of decisions on 
the timing and conditions of release that will best facilitate the rehabilitation of offenders 
and their reintegration into the community as law-abiding citizens.” In guiding the 
decisions of the Board, the Act also specifies “that the protection of society be the 


paramount consideration in the determination of any case.” 


NUMBER OF PAROLE HEARINGS 


Over the last 10 years, the number of board hearings relating to decisions in granting or 
denying parole have declined significantly, as the following table shows. 


Number of Parole Hearings, 1991/92—2000/01 
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Source of data: The Ontario Parole and Earned Release Board 


The number of parole hearings involving granting or denying decisions has declined by 
66% from about 6,600 in 1993/94 to fewer than 2,100 in 2000/01. A general decline in 
crime rates and the introduction in 1996 of conditional sentencing, which allows offenders 
to serve their sentences under supervision in the community, have reduced the number of 
offenders sentenced to Ontario correctional institutions. However, the decline in the crime 
rate has been gradual, and the use of conditional sentencing stabilized shortly after its 
introduction. Thus, these two factors alone do not account for the sharp reduction in the 
number of hearings in the last few years. 


According to studies conducted by the Board, a more important factor is that a significant 
number of offenders is not even interested in parole. Under current legislation, inmates with 
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sentences of less than six months must apply for a parole hearing in writing, whereas those 
serving longer sentences will automatically have a hearing scheduled, unless they waive the 
right to a hearing in writing. Since 1995, the monthly number of waivers signed by 
offenders has often been the same as or greater than the number of hearings completed. 


To help determine the reasons for the decline in requests for hearings, the Board 
commissioned two studies, one in 1999/2000 and the other in 2000/01. For example, the 
first study reviewed written reasons for the waivers and interviewed inmates in all regions. 
That study reported that the majority of inmates had noted on their waiver forms that they 
were either not interested in parole or that parole was a waste of time. Others indicated that 
they were receiving treatment in their respective institutions or that they had no residence 
outside those institutions. The second study consisted of interviews with inmates from one 
institution. 

Common observations of the two studies included the following: 


¢ Ministry correctional staff had not provided proper parole information to inmates as 
required by the Ministry’s Memorandum of Understanding with the Board. For 
example, the first study pointed out that not only were pamphlets and videos with 
information about parole not shown to offenders, but in a number of correctional 
institutions such pamphlets were not even available. The study indicated that “parole 
was often presented in a negative light and parole hearings were not encouraged, but 
instead parole waiver forms were often on hand ready for completion.” 


¢ Inmates reported that they felt they had little chance of having a fair and unbiased 
parole hearing. Accordingly, many did not want to go through the motions of a hearing. 


In addition to the increase in the number of waivers from long-term offenders (those serving 
six months or longer) that these studies focused on, we noted that there was also a significant 
problem in the processing of applications for hearings from short-term offenders (those 
serving less than six months). 


In Ontario, over 85% of inmates generally serve sentences of less than six months, and on 
average, provincial inmates are sentenced to only about 70 days. The majority of short-term 
inmates are not considered by the Board to be violent or high risk and would benefit from 
appropriate supervision and treatment programs in the community. 


However, our audit indicated that less than one per cent of Ontario’s short-term offenders 
had received parole hearings in recent years. Board management indicated that it was not 
practical to hold hearings for inmates with sentences of less than 90 days, as the Board might 
not have enough time to gather all the information necessary for the hearings. We examined 
applications for parole in the three regions we visited and found that applications had not 
been considered consistently across the province. Specifically, we noted the following: 


°  Inone region, the majority of short-term offenders who applied for a parole hearing 
were denied one. The senior board member for the region informed us that in that 
region it was a matter of practice to deny any hearing applications from inmates serving 
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122 days or less. Consequently, a significant number of offenders who might have 
benefited from parole were not given the opportunity to have their cases heard. 


¢ Another region could not identify short-term offenders whose applications for a parole 
hearing had been denied because it did not keep a complete record of denied 
applications. 


Under current legislation, inmates not granted parole are released after serving two thirds of 
their sentences, while those granted parole must serve the full length of their sentences with 
one third in the institution and two thirds in the community under supervision and 
conditions set by the Board. Effectiveness in promoting the safety of society requires the 
Board to help as many low-risk offenders as possible successfully reintegrate into the 
community by controlling the timing and conditions of their release. 





Recommendation | 
To more effectively control the timing and conditions of release of inmates, the 
Board should: 


e work with the Ministry to ensure that correctional institutions provide 
inmates with proper information about parole; and 


e review regional practices to ensure that consistent and equitable access is 
provided to offenders applying for parole hearings. 


Board Response 


The proposed restructuring of the Board will place senior board members and 
case officers directly in the most populated institutions and will lead to 
inmates receiving better information about parole and earned release 
programs. 


The corporate office will review the information packages given to provincial 
inmates to assure completeness, currency, and accuracy. 


The Board will specify and reiterate expectations for the timeliness and 
accuracy of information given to inmates on admission by the Ministry and for 
the quality, timeliness, and relevance of information provided to the Board by 
institutional employees with respect to parole applications, certain temporary 
absence requests, and eventually earned remission awards. 





Case officers at major institutions will be available to answer specific 
questions posed by inmates about any aspects of the parole process. They will 
also ensure inmates receive adequate information about parole and temporary 
absence applications. 


The memorandum of understanding (MOU) between the Ministry and the 
Board will be rewritten to clearly specify the Board’s expectations with respect 
to the quality and timeliness of information received by inmates about parole 





on admission. Furthermore, the MOU will underscore the high expectations | 
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that the Board holds for the receipt of timely, complete, and accurate 
information from the Ministry and the manner in which ministry employees 
process applications for parole. 


Board policies and procedures will be substantially reviewed to provide clear 
direction to board members and employees with respect to a single, consistent 
approach for processing applications for parole. These policies will also 
provide clear direction on how records of both approved and denied cases are 
maintained, collected, and reported. The performance objectives of board 
members and employees will be adjusted to include clear measures of their 
responsibilities in processing parole applications. The corporate office will 
implement a regime to regularly assess the manner in which parole 
applications are reviewed and responded to by the board members and 
employees and correct any shortcomings. Board training policies and 
programs will be reviewed and adjusted, where required, to provide for 
appropriate initial and ongoing training for board members and employees on 
their roles and obligations for processing applications for parole. 




















| VFM Section 3.09 | 


PAROLE DECISIONS AND THEIR IMPACT ON 
REOFFENDING 


A parole board’s most important function is to decide whether inmates should be granted 
or denied parole. All parole decisions are governed by legislated criteria, stipulated by the 
federal Corrections and Conditional Release Act, which require the determination of 


whether: 


¢ an offender presents an undue risk to society by reoffending before sentence expiration; 
and 


¢ release would contribute to the protection of society by facilitating reintegration of the 
offender into the community as a law-abiding citizen. 


Thus, in any given year, the parole grant rate reflects a parole board’s application of 
legislative criteria in assessing the risk offenders pose to the community, as well as the 
availability of community supports and resources for the reintegration of the offenders. 
Community supports and resources include the adequacy of supervision by parole officers 
and other community programs and resources that enable a gradual release of inmates into 
society. 

Between 1993/94 and 2000/01, the number of offenders granted parole declined from 
over 3,800 to fewer than 600. Parole grant rates have been steadily declining since 
1993/94, from 59% in 1993/94 to 28% in 2000/01. The decline in the number of 
hearings combined with the drop in parole grant rates has resulted in an 85% reduction in 
the number of offenders granted parole over the last seven years. 


i 
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The chart below is a comparison of Ontario’s parole grant rates with those of other 
Canadian jurisdictions. Statistics were not available from B.C. Senior board management 
informed us that Quebec’s parole board rates of parole were not suitable for comparison 
with Ontario’s because its parole program policies differ from those of Ontario. 


Parole Grant Rates Across Canada, 1996/97—2000/01 


707 — 


60 4 











50 4 





mem N a a iia ae 





cease = 
= mel 
= 40 a pn 
= 
o 
2 
® 
Qa 
~ 304 Va" Sige eer nts cant a 
20 4 





—&— Provinces administered by National Parole Board 
~{\~ Federal inmates 
10 5 —@— Ontario 




















1996/97 1997/98 1998/99 1999/2000 2000/01 


Source of data: The Ontario Parole and Earned Release Board 
and the National Parole Board 


Despite the fact that the same legislated criteria are used for all parole decisions in Canada, 
Ontario's parole grant rates were significantly lower than the rates for other provinces and 
even lower than rates for federal offenders, who are deemed in general to be of significantly 


higher risk. 


Ministry data show that the reoffending rate for provincial inmates released without parole 
is about 60% within two years of their release. Board data on parolees released since 
1995/96 indicated two-year reoffending rates ranging from 18% to 23% over those years. 
Research by the Ministry indicated that with support and intervention in the community, a 
significant reduction in the rate of reoffending is achievable. This research also indicated 
that a lack of community support and intervention with low-risk offenders could actually 
increase the reoffending rate. These research findings are in accord with the Corrections and 
Conditional Release Act requirement that parole boards “make the least restrictive 
determination consistent with the protection of society.” 


We reviewed Ontario’s statistics for the last 10 years for reoffending during parole to 
determine whether the significantly lower parole grant rates since 1993/94 resulted in 
corresponding reductions in the rates of reoffending during parole. Available parole 
statistics from the Board for the last 10 years are outlined in the following table. 
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Ontario Parole Grant Rates 
and Reoffending Rates, 
1991/92—2000/01 





























Source of data: The Ontario Parole and 
Earned Release Board 


As the table indicates, while Ontario’s parole system has been releasing relatively fewer 
offenders on parole in recent years, proportionally more of those parolees have reoffended 


during parole compared with the rates of reoffending from before 1994/95. 


Reoffending by parolees during parole depends on the motivation of parolees and many 
other factors, such as proper application of parole criteria by the Board in assessing risks and 
community resources and the effectiveness of parole officers in monitoring and supervising 
parolees. However, the Board needs to address the reasons for Ontario’s combination of 
lower parole grant rates and higher rates of reoffending during parole. 





Recommendation 


To better protect society through the appropriate release of inmates under 
parole supervision and conditions, the Board should conduct a systemic 
review of board decision-making to determine why parole grant rates have 
significantly decreased since 1993/94 and why, since that same time, there has 
been a general increase in rates of reoffending during parole and take 
corrective action where appropriate. 








Board Response 


The hallmark of the Board’s independence is that, through the active 
participation of Board members drawn from communities across the province, 
it renders thorough and independent judgments on individual parole 5 | 


Es 
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the Board’s decision-making and review processes. 
Ee 
















applications based upon a disciplined and methodical assessment of the 
information placed in its care. The parole grant rate reflects the collective view 
of a group of well-informed members who assess risk in the context of their 
interpretation of public safety. 


Nonetheless, the Board intends to conduct a review of the work of the federal 
and other provincial boards of parole to determine whether there are best 
practices that the Board could adopt for its own. 


Reoffending rates are a matter of ongoing interest to the Board and will be 
monitored and analyzed to illuminate board policy. In-depth analysis of 

reoffending cases will be done immediately with the intent of generating policy, 
learning, accountability, and procedural remedies to ensure best practices in 








PERFORMANCE MEASURES 


The Board has established in its Business Plan three key performance measures for 2001/02 
as outlined in the following table. 


Board Performance Measures, 2001/02 


2001/02 
Commitments 


Goals/Outcomes Measures Targets/Standards 


Reduction in Percentage of parolees | Maintain rate of Meet or 
reoffending by reoffending while on reoffending while on exceed target 
parolees parole parole at 4.5% or less 











Percentage of parolees | Maintain rate of Meet or 
suspended for serious suspension for serious exceed target 
reoffending reoffending at 2% or less 








Percentage of parolees | Maintain rate for parolees | Meet or 
reoffending within two reoffending within two exceed target 
years of release years of release at 25% or 

less 





Source of data: The Ontario Parole and Earned Release Board 


We noted that the Board has set performance targets below those already achieved; thus, 
those targets are not conducive to improving performance. Specifically: 


¢ The rate of parolees reoffending has never reached as high as the 4.5% established as 
the performance target for 2001/02. Achieving the 4.5% performance target would 
actually entail a worsening of performance, as it would be significantly above average 
historical rates. 


¢ The performance target for serious reoftending is to maintain a rate of 2% or less. 
Historically, level I reoffending in Ontario has not been above 1.5%. 
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¢ Regarding the target of maintaining the rate of reoffending by parolees within two years 
of release at 25% or less, board statistics for the last five years show that Ontario’s two- 
year reoffending rate has fluctuated from a low of 18% to a high of 23%. 








Recommendation 


To improve performance for reducing reoffending rates of parolees and 
thereby enhance public safety, the Board should set performance targets 
based on its own best results as well as those from other jurisdictions. 


Board Response 


The Board intends to study the performance measures used in other 
jurisdictions and will establish measures that it believes enhance public safety. 











PAROLE DECISION-MAKING 


Obtaining Relevant Information 


The federal Conditions and Conditional Release Act requires “that parole boards take into 
consideration all available information that is relevant to a case, including the stated reasons 
and recommendations of the sentencing judge, any other information from the trial or the 
sentencing hearing, information and assessments provided by correctional authorities, and 
information obtained from victims and the offender.” 


To comply with the legislation, the Board has established various information requirements 
to assist members in determining whether the release of an inmate would present undue risk 
to the community. More stringent requirements are imposed for the release of level I 
offenders, including confirmation of inmates’ post-release travel plans outlining how an 
offender intends to travel to a destination in the community from the correctional 
institution. Our audit, however, revealed that the required information was often not 
available for use in cases of level I offenders. 


Specifically, in over half of the parole cases we examined, the Board did not obtain all 
relevant offender information before rendering parole and other release decisions. Of the 
cases we examined, 47% involved level I offenders who were granted parole. Of those level I 
cases, 79% were missing some of the key information, such as stated reasons and 
recommendations of the sentencing judge or the travel plans of the offender. 


Our discussion with management indicated that stated reasons and recommendations of the 
sentencing judge did not constitute a core document relevant to a case. Consequently, 
police occurrence reports and Crown prosecutors’ synopses were allowed to be used as the 
equivalent of the stated reasons and recommendations of the sentencing judge. However, 
occurrence reports and Crown prosecutors’ synopses are not equivalent to the stated reasons 
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and recommendations of the sentencing judge because they are essentially evidence and 
arguments used to convict an offender whereas the stated reasons and recommendations of 
the sentencing judge have taken all objective evidence and circumstances into consideration. 
The use of only occurrence reports and prosecutors’ synopses may reinforce the perception 
that sources of information used by the Board are biased. 


In addition, board failure to review travel plans of level I offenders before releasing them 
imposes unnecessary risk on the community. 








Recommendation 


_ To provide a better basis for granting parole, the Board should receive and | 
consider all information necessary to support its parole decisions, including | 

_ the stated reasons and recommendations of the sentencing judge and | 
offenders’ travel plans in all cases involving higher-risk offenders. 


Board Response 


At the present time, judges do not always provide reasons for sentencing and 
their recommendations. If the courts are prepared to make stated reasons and 
recommendations available, the Board will consider them. It is the Board’s 
intention to pursue an understanding with the Ministry of the Attorney General © 
and the Ministry of Public Safety and Security to ensure that the stated | 
reasons and recommendations of the trial judges are reliably and routinely 
forwarded to ministry officials, thus permitting them to be considered as a core 

_ document in the future. | 


The Board also asserts that the agreed statement of facts between the Crown 
and the defence that the judge also sees is an important and informative 
document that is available from the court. The Board intends to obtain this 
document from the Attorney General for inclusion as a core document for all 
parole cases. 





Setting Parole Conditions 


In accordance with the Ministry of Correctional Services Act, the Board has established 
standard conditions for all parolees to comply with, except where board members 
determine the deletion of any such conditions to be reasonable. These standard conditions, 
as set out in the Ministry of Correctional Services Act, Regulation 778, section 48, are as 
follows. 


It is a condition of every grant of parole, unless the Board orders otherwise, that the 
parolee shall, 


(a) remain within board jurisdiction; 


(b) keep the peace and be of good behaviour: 
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(c) obtain board or parole officer consent to change residence/employment; 
(d) report as required to parole officer and local police; and 
(e) refrain from associating with anyone engaged in criminal activity or, unless 
approved by a parole officer, anyone with a criminal record. 
To aid parolees’ reintegration and reduce risk to the community, board members may also 
add special conditions based on case-specific facts. 


Our review indicated that board members had properly imposed standard conditions as 
required by board policy. However, the Board could not provide evidence of the reasons for 
not imposing specific conditions of parole recommended by parole officers or police (for 
example, no contact with the victim, no possession of firearms, or submission of passport). It 
is board practice to audiotape hearings but not the case discussions or the decision-making 
portions of its hearings. 








Recommendation 


To help ensure public safety, the Board should appropriately support its 
decisions not to impose special parole conditions recommended by police or 
parole officers. 


Board Response 


The Board will direct board members to indicate on the parole consideration 

form their knowledge of recommendations from police or parole officials, and, 
in instances where the Board does not intend to apply the recommendation(s), 
provide a rationale for that decision. “a 











QUALITY ASSURANCE 
AND PERFORMANCE REVIEW 


The Board has established a process for internal review of parole files and corresponding 
audiotapes of hearings to monitor the quality of parole decision-making and to determine if 
there are any training or policy issues that require attention. This process is designed to be 
proactive, and requires the examination of randomly selected parole-applicant cases on a 
quarterly basis to identify trends and issues within the parole system. Senior board members 
responsible for these reviews are required to maintain a record of the randomly selected 


parole files, including audiotapes. 


In addition, all parole cases resulting in serious, level I incidents during parole are reviewed 
jointly with the Ministry. The Deputy Minister and the Chair of the Board can also request 


special investigations. 


We noted that internal reviews were performed for all cases involving level I reoffending in 
2001/02. However, we also noted significant non-compliance with the requirement for the 
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quarterly random file reviews. Specifically, of the four regions, only one was performing the 
reviews. Another region had started the review process in October 2001. 


In cases where board reviews were done, the results indicated weaknesses or problems in the 
following areas: 


¢ poor interviewing skills; 
¢ inappropriate setting of special conditions of parole; and 
¢ inadequate consideration of factors for and against parole. 


However, despite having identified problems, the results of reviews were not used to take 
action to improve the performances of board members. For example, additional training, 
where required, was not provided to enhance members’ effectiveness. 





Recommendation 


To improve the quality of its members’ decision-making and overall board 
performance, the Board should systematically monitor the parole decision- 
making process and take corrective action, including the provision of 
additional training, where necessary. 


Board Response 


The Board endorses this recommendation. The Board is conducting a 
thorough review of the present board decision-making processes to provide a 
reliable, rigorous, and transparent regime for random reviews that both the 
senior members and the corporate office can employ to measure the work of 
panels. 


Furthermore, the Board is concurrently examining processes for completing 
internal reviews in cases where a reoffence has occurred. This work will form 
the basis for reviews of our present policies, procedures, and training 
programs. 











SELECTION AND APPOINTMENT 
OF BOARD MEMBERS 


According to the Corrections and Conditional Release Act, to ensure that legislative criteria is 
properly applied, parole boards should “adopt and be guided by appropriate policies” and 


provide their members “with the training necessary to implement those policies.” 


‘To comply with that legislative requirement, the Board provides a mandatory basic training 
program for all new members consisting of several three-day courses within the first year of 
appointment. Subsequent seminars and workshops are held periodically in conjunction with 
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regional board meetings. Attendance by members at special training and conferences is 
encouraged but not required. 


Our review indicated that board members were being provided with proper initial 
orientation and basic training to fulfill their responsibilities. However, basic training is 
effective only when members understand the concepts of parole and have the appropriate 
knowledge, abilities, skills, and commitment to excellence in this area. In that regard, we 
were unable to determine whether the current appointment process was selecting the most 
suitable members. 


Board members are appointed by the Public Appointments Secretariat of Management 
Board Secretariat. Currently, the Board has little influence in the selection of its new 
members. Selection criteria for members of the Ontario Board are similar to those for the 
National Parole Board. However, unlike the National Parole Board, the Ontario Board is 
not involved in the initial screening of applicants, and there is no formal process for the 
Board to provide input regarding the abilities, skills, commitments, and suitability of 
applicants. We noted that, currently, the process for selecting candidates for the Ontario 
Board does not include input from board members whereas the selection process for 
membership to the National Parole Board involves a selection committee that includes both 
board and external representatives. 








Recommendation | 


To ensure that the most suitable candidates are selected and appointed as 
board members, the Board should work with the Public Appointments 
Secretariat of Management Board Secretariat to establish a more formal 
process for assessing the abilities, skills, commitments, and suitability of 
applicants for board membership. 


Board Response 


Work has already commenced between the Chair of the Board and key 
government officials to produce a thorough, formal, and rigorous process for 
identifying, selecting, training, and assigning board members, as well as a 
better process for assessing the performance of board members and, where 
necessary, providing remedial counselling or training. 





Recommendations for the candidate’s suitability to serve on the Board will 
then be made to the Public Appointments Secretariat of Management Board 
Secretariat. 
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CASE-RELATED DECISIONS 


In addition to making decisions granting or denying parole, the Board also makes case- 
related decisions, such as the approval of parolees’ changes in employment or residence, the 
revocation of parole, and the authorization of warrants. A full-time member generally makes 


these decisions at the regional offices. 


Parolees are supervised by parole officers in the community. When a parole condition is 
breached, parole officers report to the Board for further direction. The Board may revoke 
parole as a result of a breach of condition or a new offence. 


Our review indicated that the Board was timely in discharging its responsibilities and taking 
appropriate actions. More specifically, we noted the following. 


¢ The Board ensured that it received reports on the progress of parolees from probation 
and parole officers and reviewed these reports on a timely basis. 


¢ When parole conditions set by the Board were subsequently found to be unsuitable for 
a particular parolee, those conditions were modified or changed to ensure public safety 
and aid the parolee’s rehabilitation efforts. 


¢ Revocations and warrants were issued on a timely basis when there were breaches of 
parole conditions that presented an undue risk to community safety. 
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MINISTRY OF TOURISM AND RECREATION 


3.10—Tourism Program 





BACKGROUND 


The Tourism Program of the Ministry of Tourism and Recreation is responsible for 
developing and promoting Ontario’s tourism industry. This industry includes a number of 
businesses that provide services like accommodation, food, attractions, transportation, retail 
opportunities, and travel assistance. The Ministry estimates that these businesses employ 
approximately 500,000 people and that the industry is the province’s largest employer of 
young and seasonal workers. Annually, tourism generates an estimated $17 billion for the 
Ontario economy and $2.5 billion in provincial tax revenue. 


The objectives of the Tourism Program are to promote tourism; stimulate employment; 
support the use of parks, tourist facilities, and attractions in Ontario; and encourage 
improvement in the standards of accommodations, facilities, and services offered to 
travellers. The role of the Ontario Tourism Marketing Partnership Corporation 
(Corporation)—a ministry agency that was established by regulation under the Development 
Corporations Act and began operating in 1999—is to market Ontario as a tourist 
destination by developing and implementing tourism advertising and marketing plans and 
enhancing partnerships with the tourism industry. 


The Tourism Act and the Ministry of Tourism and Recreation Act provide the legislative 
authority for the operation of the Tourism Program. The Ministry operates 18 tourist 
information centres and is responsible for a total of nine tourism agencies and attractions. 
The Ministry directly operates the tourism attractions Huronia Historical Parks and Old 
Fort William. The tourism agencies, in addition to the Corporation, are governed by their 
own legislation and generate revenue to pay for the majority of their own expenses. These 
agencies are: Ontario Place, Metropolitan Toronto Convention Centre, Ottawa Congress 


Centre, Niagara Parks, St. Clair Parks, and St. Lawrence Parks. 


For the 2001/02 fiscal year, tourism operating expenditures totalled $83 million, of which 
$52 million was spent by the Corporation and $11 million was provided to the other 
tourism agencies. 


See 
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AUDIT OBJECTIVES AND SCOPE 


The objectives of our audit of the Tourism Program were to assess whether the Ministry and 
the Ontario Tourism Marketing Partnership Corporation had adequate procedures in place 
to: 


¢ measure and report on the effectiveness of the program in promoting tourism in 
Ontario; 


* ensure compliance with legislation as well as government and ministry policies; and 
* manage resources with due regard for economy and efficiency. 


The criteria used to conclude on our audit objectives were discussed with and agreed to by 
senior management at the Ministry and at the Corporation and related to systems, policies, 
and procedures that the Ministry and the Corporation should have in place. 


The scope of our audit, which was substantially completed in March 2002, included 
discussions with ministry and corporation staff, as well as the review and analysis of 
documentation maintained at their offices and the office of Management Board 
Secretariat's Shared Services Bureau, which is partly responsible for the acquisition of 
consulting services. In addition, our audit work related to tourism agencies and attractions, 
with the exception of the Corporation, was limited to ministry funding for the maintenance 
and repair of capital assets. 


Our audit was conducted in accordance with the standards for assurance engagements, 
encompassing value for money and compliance, established by the Canadian Institute of 
Chartered Accountants, and accordingly included such tests and other procedures as we 
considered necessary in the circumstances. 


We also reviewed the relevant work performed by the Ministry’s Internal Audit Services, 
which had undertaken a review of the Ministry’s accountability framework with its agencies. 
This review was useful in reducing the extent of our audit work. 


OVERALL AUDIT CONCLUSIONS 


In the past decade, the Tourism Program has experienced numerous shifts in focus and 
organizational structure—as evidenced by its five different mission statements— as the 
responsibility for the Program has changed ministries six times. Another major change was 
the recent creation of the Ontario Tourism Marketing Partnership Corporation, which was 
delegated the responsibility for tourism marketing and advertising. In addition, at least 11 
provincial ministries have a tourism objective as part of their mandates. Given the 
significance of tourism to the Ontario economy, the Ministry needs to take a more proactive 
leadership role in developing and implementing a long-term tourism strategy to help 
co-ordinate the many activities of public- and private-sector organizations that contribute to 
the promotion of tourism in Ontario. 
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Overall, we concluded that the Ministry and the Corporation did not have sufficient 
procedures to measure and report on the effectiveness of the Program in promoting tourism 
in Ontario. As well, in its annual business plan, the Ministry did not report the actual results 
achieved for any of its previously published performance measures. Also, after three years of 
operation, the Corporation had not submitted an annual report to the Legislature as 
required. 


We also found that the Ministry and the Corporation did not have adequate procedures in 
place with respect to several aspects of their operations to ensure that the Program was 
delivered with due regard for economy and efficiency. Specifically, we noted the following: 


I 


Tourism publications were not sufficiently comprehensive and were not published on a 
timely basis. For example, the Corporation’s main tourism guide lists only 1,400 of an 
estimated 8,000 tourism facilities, and the 2001/02 winter events guide was not 
published until December 2001 and contained listings for events that had already taken 
place. 


There was no process in place to collect information on the tourism-related activities 
undertaken by other ministries and on the financial support provided by other 
ministries to the tourism industry. As a result, there was a risk of overlap and duplication 
of tourism-related programs and services offered by the government’s ministries and 
agencies. 


Ontario’s accommodation rating system and quality assurance standards for 
accommodations have not been adequately co-ordinated between ministries and the 
private sector. As a result, Ontario is one of only a few leading tourist destinations that 
does not have province-wide quality assurance standards or an accommodations rating 
system. 


Based on our examination of a sample of consulting contracts, we found that 
management consulting services were in many instances not justified by a business case, 
nor were related contracts signed on a timely basis. Moreover, several contracts were 
awarded directly to the vendor without competition, and other projects were split into 
separate contracts, thus allowing the Ministry to avoid open competition requirements. 


Both the Ministry and the Corporation maintain separate tourism information 
databases, and they do not have a process in place to periodically compare and update 
the information contained in these databases. 
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DETAILED AUDIT OBSERVATIONS 
STRATEGIC PLANNING AND REPORTING 


Ministry Vision and Strategic Planning 


An organization’s vision or mission statement identifies its overall purpose, provides general 
direction, and is important for developing specific plans for implementation. The 
development of a ministry vision statement assists in defining how services are to be provided 
and helps in the development of quality programs and services with greater efficiency and 
effectiveness. 


Over the past 10 years, responsibility for the Tourism Program has changed ministries six 
times as a result of government reorganizations. Each reorganization has resulted in changes 
to the vision for tourism to reflect the focus of the new ministry. The following table 
illustrates some of the changes in the vision for tourism since 1993. 


Tourism Program, 1993-2002 


Ministry Responsible _ Period Vision or Mission Statement 


Tourism and Recreation To 1993 To “enhance the economic and social well- 
being of residents of Ontario by ensuring the 
development and maintenance of services, 
programs and facilities to support a wide 
range of tourism and recreation opportunities 
and choices in Ontario for residents and 
visitors.” 





Culture, Tourism and 1994-96 To “promote accessibility, economic 

Recreation advancement and development of culture, 
information management, tourism and 
recreation.” 





Economic Development, 1997-99 To help Ontario “achieve sustainable growth 

Trade and Tourism and jobs and to position Ontario as a leading 
North American destination for domestic and 
international travellers.” 





Tourism 1999-2000 To be “an innovative and dynamic tourism 
industry” that: “raises Ontario’s profile around 
the world as a must-see tourist destination; 
encourages visitors to stay longer and return 
more often; shares the most accessible, 
up-to-date information with potential tourists; 
offers world-class tourism facilities and 
experiences for visitors; boasts the best 
trained, friendliest staff in the world; and 
generates jobs and develops the full 
economic potential of every region of the 
province.” 


Tourism, Culture and 2000-02 To be “a recognized national leader in job 
Recreation creation and investment through the support 
and development of the tourism...sector.” 


Tourism and Recreation From April 2002 | To be determined. 














Source of data: Ministry of Tourism and Recreation 
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As indicated in the table, the Tourism Program’s vision or mission statements have changed 
with each successive ministry realignment. Normally, mission statements and their related 
goals do not change unless the circumstances under which they were created have changed. 
The Tourism Program has experienced significant changes in organization and focus over 
the past 10 years, and with each realignment, management structures within the Tourism 
Program have also been subject to numerous changes. In addition, in 1999, the 
responsibility for tourism marketing was transferred to a new agency, the Ontario Tourism 
Marketing Partnership Corporation. 


In 1999, the then Minister of Tourism announced that, “our Ministry will co-ordinate the 
development of the province’s first comprehensive tourism strategy.” The anticipated 
strategy was developed but was never formally approved. In reviewing this document, we 
observed that it contained measurable short- and long-term objectives and an outline of 
specific activities to be completed. Given the complexity of the tourism industry and its 
significance to Ontario’s economy, there is a need for an overall strategic plan that would 
provide program staff with clear direction that transcends organizational restructuring, 
outlines accountability for program responsibilities, and prioritizes the Program's many 
objectives. 








re 







Recommendation _ 


To help strengthen accountability and provide clear direction to fulfill its vision, 
the Ministry should develop a strategic plan for the Tourism Program that has 
measurable short- and long-term objectives and an action plan with a defined 


accountability framework. 


Ministry Response _ 





The Ministry agrees with the recommendation and proposes to develop a 
renewed Tourism Strategic Pian that includes measurable short- and long-term 
objectives and a defined accountability framework. 











Procedures to Measure and Report on Program 
Effectiveness 


Management Board of Cabinet directives require that ministries publish a combined 
business plan and annual report that outlines plans for the following year and reports on 
performance from the previous year. The purpose of the government business planning 
process is to improve decision-making, support the development and delivery of high- 
quality programs, ensure outcomes are aligned with overall government priorities, and 
improve accountability for measuring and achieving results. Published business plans and 
annual reports are also intended to support openness and accountability to the public and 
the Legislature. 


EEE 
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We reviewed the Ministry’s business planning process for the 2000/01 and 2001/02 fiscal 
years and noted that the Ministry outlined a number of performance measures, each having 
targets and commitments for its core businesses. Although some actual results were reported 
internally, the Ministry did not report on the results of any of its performance measures in its 
published business plans. Therefore, the intent of the business planning process to support 
openness and accountability to the public and the Legislature for the achievement of 
program objectives was not fulfilled. 


We also noted that the Ministry’s performance measures and related commitments were 
often high-level indicators of Ontario tourism trends. For instance, one of its 2001/02 
commitments was to generate $500 million in total incremental tourist spending. This 
measure does not indicate the extent to which ministry activities increased tourism spending. 
Such high-level measures are affected by a variety of factors outside the Ministry’s control, 
such as currency exchange rates, international events, and domestic economic conditions. 
Furthermore, local municipalities, private organizations, independent businesses, and several 
other provincial ministries also promote tourism in Ontario. With multiple organizations 
promoting different aspects of tourism, It Is difficult to isolate the Ministry’s contribution to 
achieving the stated goals of the Tourism Program. Performance measures that relate to the 
Program’s activities needed to be developed to isolate the Ministry's contribution to the 
overall objective of promoting tourism. 


The Ontario Tourism Marketing Partnership Corporation recognized the need to measure 
and report on the achievement of its objectives and established the Marketing Performance 
Assessment Committee in 1999. In July 2001, a performance report on the Corporation's 
operations was prepared for the committee. We observed that the Corporation had 
developed a number of performance measures. However, the Corporation needs to develop 
specific standards or expectations for many of these performance measures. For example, 
many measures are based on specific activities, such as the number of calls received or the 
number of publications distributed. Activity reporting without comparison to an established 
benchmark or expected outcome makes it difficult to determine whether the reported 
activity represents success or failure. 


In addition, for the Corporation to be accountable for its expenditure of public funds, it is 
required to prepare an annual report on its affairs to be tabled in the Legislature. However, 


after three years of operation, the Corporation had still not prepared an annual report for 
the Legislature. 


We observed that other jurisdictions had developed performance measures and were 
annually reporting on their achievements. They were also providing explanations as to why 
key measures were not met. For instance, in 2001, another Canadian province publicly 
released an annual report on tourism that contained a number of performance indicators 


including comparisons of targets with actual achievements, along with an analysis of 
achievements or reasons for not achieving each target. 
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Recommendation 





To provide better accountability to the public and the Legislature for their use 
of public funds and to encourage a results-based operational focus: 


e the Ministry should develop performance measures over which it has a 
reasonable degree of influence, report on the actual achievement of these 
measures, and provide explanations for any significant deviation from the 
expected outcomes; and 


e the Ontario Tourism Marketing Partnership Corporation should develop 
standards for its performance measures and prepare the required annual 
reports for submission to the Legislature. 


Ministry and Corporation eeponve | 


The Ministry agrees that some of its performance measures are out of its 
sphere of influence and will develop additional measures that are more directly 
influenced by ministry programs. Both commitments and results for these 
measures will be reported. 
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The Ontario Tourism Marketing Partnership Golporation supuoite the 
development of an effective method of measuring performance. The 
Corporation will report on its performance in its 2003/04 business plan. The 
Corporation will develop standards for its performance by end of fiscal 
2002/03, by researching other jurisdictions of comparable mandates and 
budgets to establish benchmarks. The Corporation currently employs a return- 
on-investment model, which uses ad tracking to assess the effectiveness of its 
advertising and measures ‘performance of this aspect of marketing. The 
Corporation is also building in performance tracking as part of the 
implementation of a new planning and accountability framework. We will 
implement these standards for the 2003/04 fiscal year and monitor them 
throughout the year to ensure they provide results-based information. 


The Ministry will work with the Corporation to ensure that it prepares the 
required annual reports for the Minister’s approval and submission to the 
Legislature. - a 


Be 




















Co-ordination of Tourism Initiatives 


The Ministry and the Corporation are not the only promoters of tourism in the province: 
many other public and private organizations and businesses promote tourism in Ontario. 
For example, the Ministry indicated that there were at least 11 other provincial ministries 
that have a tourism objective in their mandates. 


In the past there have been attempts to co-ordinate the various tourism-related activities 
administered and funded by different ministries and agencies. For instance, in 1997, an 
inter-ministerial committee comprising twelve ministries with a tourism mandate was 
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created to analyze policy issues related to tourism. During its two years of operation, the 
committee produced a draft “Tourism Action Plan” for submission to Cabinet but the plan 


was never submitted. 


At the time of our audit, the Ministry was using a wide range of approaches to co-ordinate 
rourism-related activities. For example, ministry staff attended meetings with many of the 
major tourism associations, and corporation personnel conducted province-wide “town hall” 
meetings to discuss marketing initiatives. Policy was co-ordinated through a number of 
frmal and informal means with staff from other ministries and agencies. As well, the 
Ministry was regularly commenting on submissions that had a tourism component and were 
prepared by other ministries and agencies for government committees. However, the 
Ministry did not have adequate information on the tourism-related programs administered 
by other ministries and agencies or on the related funding provided to the tourism industry 
by these ministries and agencies. 


The Ministry stated that most other provincial programs provide funding to specific 
industries or regions with tourism as a secondary objective. For example, the Ministry of 
Northern Development and Mines provides assistance to support northern communities, 
and the Ministry of Agriculture, Food and Rural Affairs provides assistance to rural areas. In 
fact, each of these ministries provide more direct financial assistance to the tourism industry 
than the Ministry of Tourism and Recreation. The Ministry therefore needs be aware of the 
financial support the tourism industry is receiving from other provincial ministries to 
prevent possible overlap and duplication. 


Given the number of different approaches and programs supporting the tourism industry, 
there is a need for more proactive leadership in the co-ordination of tourism activities. For 
effective co-ordination, roles and responsibilities need to be well defined. The lack of clear 
responsibilities can lead to inefficiencies in program delivery. For example, in 1998, the 
Minister of Northern Development and Mines announced the creation of the Northern 
Ontario Tourism Marketing Association (Association) with the objective of developing 
tourism programs to market Northern Ontario. From the outset, the Corporation and the 
Association experienced difficulties co-ordinating their activities, and the intended 
integration of their marketing plans and customer information systems was never 
accomplished. The Ministry informed us that the Association is no longer in operation. 


Given that the activities of various ministries and agencies can contribute to the Ministry's 
overall objective of promoting tourism, it is important for the Ministry to co-ordinate its 


activities with other government and industry programs to ensure that Ontario tourism is 
effectively marketed. 









Recommendation 


To help ensure that provincial funding in support of Ontario’s tourism industry 
is used in an efficient and effective manner, the Ministry should: : 
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¢ minimize the risk of overlap and duplication between its programs and 
services and those of other ministries and agencies; 

¢ develop a process to collect financial information on the support provided 
to the tourism industry by other ministries; and 

e develop a strategy to assist in the co-ordination of all government activities 
that Promote tourism i in ike pale 





Ministry Response 


The Ministry | recognizes the Importance On minimizing the risk of Poverip and 
duplication between its programs and those of other ministries. The new 
Tourism Strategic Plan will oe. a focus $ for a ae aeasting government 
efforts in support of tourism. _— - 





To support ongoing co-ordination, cee Ministry wilt icdeveiog an annual survey 
in 2003/04 to collect information on the programs, services, and financial | 


_ support provided to the tourism industry by other ministries. _ | 














Legislative Review and Tourism Licensing 


The Tourism Act gives the Ministry the authority to license and regulate tourism and 
accommodation establishments. The Act outlines, for instance, how no advertising matter or 
sign connected with a tourist establishment that is subject to the provisions in the Act shall 
contain any inaccurate or misleading statements. The Ministry of Tourism and Recreation Act 
outlines the objectives of the Ministry and provides the authority to develop programs of 
financial assistance to further those objectives. 


In 1996, the Ministry reviewed the Tourism Act and explored alternatives to the licensing of 
tourist establishments, and, in 1998, legislative amendments were proposed that would 
remove the province-wide requirements of the Zowrism Act to license tourism establishments. 
These proposed amendments were considered to be an interim measure until the licensing 
of establishments that use Crown resources could be replaced by an industry self- nated 
certification process. The legislative changes were intended to better reflect the province’s 
perceived role in tourism, but the proposed changes were not made. 


The Ministry is attempting to move away from its direct regulatory role citing a lack of 
resources and expertise to properly administer and enforce the provisions of the Tourism Act: 
The Ministry also contends that revisions to the Ministry of Tourism and Recreation Act also 
need to be made to reflect the current role of the Tourism Program. 


As part of the 1998 legislative review, the Regulation to the Tourism Act was amended, 
exempting approximately 2,500 tourist establishments that do not use Crown lands from 
the requirements of the Act, including the need to obtain a licence. The Ministry would 
continue to license the approximately 1,400 tourist establishments that use Crown lands. In 
having made these regulatory amendments, the Ministry may have created inequities 
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between the tourist establishments that use Crown lands and those that do not in that only 
establishments using Crown lands must obtain a licence and comply with the many 
requirements of the Tourism Act. 


In the 2001/02 fiscal year, the Ministry planned to conduct another review of legislation, 
which would include consultation with the industry. This review 1s expected to provide a 
timely and appropriate context for examining solutions proposed for tourism licensing. As of 
March 2002, this review was still in progress. 


In addition we noted the following: 


© The Ministry informed us that approximately 290 tourist operators using Crown lands 
had not renewed their licences for the 2001/02 fiscal year. While the Ministry did not 
know why these establishments had not renewed their licences, ministry staff believed 
that most of the establishments were still operating. 


© The Ministry does not conduct any routine inspections of tourist establishments. The 
licensed establishments are required to provide for the health and safety of their guests, 
and facilities must comply with other legislation such as the Building Code Act, 1992, 
the Health Protection and Promotion Act, and the Environmental Protection Act. We 
were informed that the operator is relied on, as a condition of the licence, to ensure 
compliance with the regulatory requirements of the Zourism Act. The Ministry did not 
have a process in place to periodically ensure compliance with the legislation. 








: | 
Recommendation 


To clarify the Tourism Program’s responsibilities and ensure compliance with 
relevant legislation, the Ministry should: 


° consider reviewing all other tourism legislation during its current review of 
the Tourism Act, and 


° follow up on establishments that have not renewed their licence and review 
procedures to ensure that tourist establishments comply with the 
legislation currently in place. 


Ministry Response 


The Ministry appreciates the value of a comprehensive approach to legislation 
affecting tourism but notes there are 92 provincial statutes and regulations 
that affect tourism in Ontario with most not exclusively related to tourism or 
administered by the Ministry. The Ministry will need to weigh the anticipated 


costs and benefits of conducting a review of those statutes and regulations to 
determine its feasibility. 


To deal with those establishments that fail to renew their licences, the Ministry 
will review the licensing function in 2003/04 with a view to removing 
impediments that deter tourism operators from complying with the legislation 
and to increasing the incentive to do so. The Ministry will work with its own : 
pale 
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field service offices and those of the Ministry of Northern Development and 
Mines to ensure there is a timely and effective follow-up with delinquent 
tourism operators. _ Sl ge : : 








TOURISM PROMOTION 


Ministry statistics indicate that the number of tourists visiting Ontario has gradually 
declined over the last 10 years. A tourist is a person who travels for leisure, business, or other 
purposes at least 40 kilometres outside their usual environment, either overnight or for 
same-day activities. As illustrated in the following bar graph, the decline results from a 25% 
decrease in the number of domestic tourist visits (including those by Ontario residents), 
which was partially offset by an increase in international tourist visits. In comparison, the 
number of visits by Ontario residents to other jurisdictions was also reported to have 


declined. 


Number of Tourist Visits to and from Ontario, 1990-99 
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Source of data: Ministry of Tourism and Recreation 


The domestic statistics for 2000 and 2001 and the international statistics for 2000 indicate 
further declines in the number of tourist visits. In spite of the overall decline in the number 
of visitors over the last 10 years, the Ministry estimates that expenditures made by tourists in 
Ontario have increased marginally. 


The overall decline in the number of people vacationing in Ontario is attributed to a 
number of factors—such as recessions and currency fluctuations—that are outside the 
direct control of the Ministry and the tourism industry. However, innovative marketing and 
advertising can attract both domestic and international visitors to the province and 
encourage Ontario residents to vacation in Ontario. 





Tourism Program 267 









| VEM Section 3.10 | 


| VEM Section 3.10 | 





we ee ee 


Marketing Plans 


The Corporation is responsible for tourism marketing. Its annual tourism marketing plan 
provides a high-level assessment of the products and services offered by the tourism industry 
‘1 Ontario, as well as the geographic areas that the Corporation will target in its marketing 
activities. The plan also outlines the resources available to the Corporation and provides 
general direction to its management for decision-making relating to marketing and the use 


of resources. 


We reviewed the Corporation’s 2001/02 and 2002/03 marketing plans and the supporting 
documentation and analyses. We observed that the published marketing plan, which is 
available to the public and industry stakeholders, provided an indication of the 
Corporation’s marketing objectives for the year, including the industry products and services 
it intended to promote. However, we observed that documentation and analysis used to 
support the marketing decisions needed to be improved. Specifically, we noted that there 
was a lack of documentation to support the decisions on the markets to be included or 
excluded from the annual marketing plan. We also found no formal analysis of the viability 
of potential demographic or geographic markets to target for advertising campaigns. 


In reviewing other jurisdictions, we noted that the federal agency responsible for tourism 
marketing for Canada had developed a market portfolio analyses process. The process was 
developed to assist in decision-making by providing a method to objectively evaluate and 
prioritize the tourism potential and performance of the target markets fairly and 
consistently. The process complements subjective market intelligence and feedback from the 
industry. The federal process indicated that a number of countries had a high potential for 
good marketing results, including countries not being targeted by the Corporation, such as 
Mexico, Taiwan, Hong Kong, South Korea, and Brazil. 





[ . 
Recommendation 


To maximize the impact of its marketing process for attracting visitors to 
Ontario, the Ontario Tourism Marketing Partnership Corporation should: 


° require the selection of potential markets be supported by documented 
analysis; and 


° review the applicability to Ontario of marketing analysis research used in 
other jurisdictions. 





Corporation Response 






The Corporation ’s annual marketing strategy will include a market portfolio 
analysis (MPA) process, based on best practices found in other jurisdictions, 
including the Canadian Tourism Commission (CTC). 


In developing its marketing strategy for 2003/04, the Corporation has reviewed 
the CTC’s MPA model, and made use of documented analysis of consumer data 
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on U.S. and domestic markets to guide its marketing direction in those 
markets. It has also used information on overseas markets based on recent 
reviews involving interviews with industry sources. 





The Corporation will continue to improve on documenting its evidence-based 
decision-making and refine its marketing plan development process in the next 
two years. 











Advertising Program 


To promote Ontario as a tourist destination, the Corporation, in association with the 
government's Advertising Review Board, engages advertising agencies to assist in the 
creation and placement of advertisements in a variety of media, including television, radio, 
and print materials. To share the cost of its advertising campaigns, the Corporation enters 
into arrangements with tourism industry partners, such as hotel chains and local tourism 
associations. [he Advertising Review Board and the Corporation have contracted with three 
advertising agencies to provide advertising services. For the 2001/02 fiscal year, the 
Corporation spent approximately $30 million on tourism advertising campaigns, which 
included special one-time funding of $12 million for an enhanced marketing strategy to 
promote Ontario as a travel destination, and the Corporation recovered $6 million from its 
industry partners. 


We reviewed the policies and procedures in place to monitor the approval and payment of 
advertising agencies. We observed a number of financial control weaknesses, some of which 
we reported in 1999 as part of a review of financial controls at the former Ministry of 
Economic Development, Trade and Tourism. Specifically, in this audit we observed the 
following: 


* Agency billings were not checked to ensure that the advertising had actually been 
completed in the manner and volume agreed to. In addition, invoices from two of the 
agencies did not specify what part of the total billing was for commissions. 
Consequently, the accuracy and completeness of advertising payments could not be 
verified by corporation staff. Contracts with two of the advertising agencies allow for the 
auditing of the agency’s billings up to four years after the expiration of the contract. 
However, these two advertising agencies’ billings had never been audited. The current 
contracts with the three advertising agencies expired between January and November 
2001 and have been extended by letters of agreement. We were informed that, 
subsequent to our audit and based on a competitive process, the Corporation negotiated 
new advertising agency agreements. 


* The contracts with the advertising agencies require that performance reviews of the 
agencies’ delivery of service be carried out annually to ensure the execution of programs 
in accordance with stated objective and policies. However, the Corporation has not 
formally reviewed the performance of its advertising agencies. 


cS a a a A a ee 
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Recommendation | : 

To help ensure the economic, efficient, and effective delivery of its advertising 
campaigns, the Ontario Tourism Marketing Partnership Corporation should: | 


© conduct or arrange for the auditing of the billings of the advertising 
agencies it contracts with to ensure that planned advertisements have been 
placed and agency billings are accurate; and 


° complete, as required, the annual performance reviews of the advertising 
agencies. 


Corporation Response 


The Corporation previously used the services of three agencies in addition to 
the agency of record for the government of Ontario—one for domestic 
advertising, one for international advertising, and one for publications. As of 
August 2002, the Corporation has consolidated its use of agencies to two—one 
for both domestic and international advertising and one for publications. 


The Corporation will engage the agency of record to be responsible for 
purchasing all media, both domestic and international. The Corporation's new 
advertising agency will only undertake media planning and production. The 
agency of record will be responsible for all other documentation and invoices. 
The Corporation is also in discussions with the Advertising Review Board’s 
executive director on the issue of periodic auditing. The Corporation will 
monitor this process and make recommendations to the Advertising Review 
Board as appropriate. 


Training will be provided to the Corporation’s management on agency 
management, including contents and terms of the agency contract. 


A formal agency performance review will be completed annually. Informal 
review sessions will take place periodically. 














Consumer Publications 


An integral part of the Corporation’s tourism marketing process is the production and 
distribution of printed promotional and informational materials. These publications are 
designed to attract travellers to Ontario and help encourage travellers to stay longer and 
spend more. Publications also provide tourist operators with a cost-effective medium to 
promote their establishments. The revenue generated through the sale of advertisements 
within publications helps reduce the Corporation’s production cost. In the 2001/02 fiscal 
year, the Corporation spent approximately $3 million to create, publish, and distribute 


about 10 publications, and the Corporation recovered $1 million from the advertisements 
placed by tourist operators. 
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We reviewed the Corporation’s policies and procedures related to the production and 
distribution of publications and observed the following: 


* Tourists had expressed concerns to the Ministry about the Corporation’s publications, 
specifically that they were not sufficiently comprehensive and were not easy to use in 
planning a trip. For example, we observed that the Corporation’s main tourism guide 
contains listings for about 1,400 facilities that had paid a fee to be included in the 
publication; however, more than 8,000 facilities are recorded in its database. In 
comparison, tourist publications in other jurisdictions, such as British Columbia and 
Nova Scotia, contain detailed information on large numbers of their provinces’ facilities, 
making these guides far more complete than those currently available in Ontario. 


* Many of the Corporation’s publications were not available on a timely basis. In some 
cases, they were not available prior to the start of the season for which they were 
intended to be used. For example, the 2001/02 Winter Events Guide was not available to 
the public until early December 2001 and contained listings of events that had already 
taken place. This publication would not be helpful for tourists wanting to preplan 
vacation activities. In comparison, other provinces generally had seasonal publications 
available to the public as much as four months prior to the start of the related season. 


* The Corporation did not offer tourist operators the option of placing advertisements in 
its French-language publications. Therefore, none of the French-language publications 
produced by the Corporation contained partnership advertisements. In contrast, the 
Corporation's English-language publications generate revenues that represent one third 
of the production cost. We noted that Nova Scotia had been successful in arranging for 
a considerable amount of advertising in its French-language tour guide. 


¢ The Corporation supplies publications to regional travel associations and municipalities 
in quantities that are based on the amounts supplied in the previous year rather than on 
an objective assessment of demand. In addition, we noted that the Corporation’s 
publications inventory contained a significant quantity of old products that are no 
longer useful to tourists and for which it continues to pay storage costs. 





Recommendation | 
To help ensure that its tourism publications are produced and distributed in an 
economical and effective manner and that they effectively meet the needs of 
tourists, the Ontario Tourism Marketing Corporation should: 


¢ review the completeness of the tourism information contained in its 
publications, and release publications on a more timely basis; 


e assess the potential for obtaining advertising revenue for its French- 
language publications; and 


/¢ review its distribution and inventory policies. 
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Corporation Response : | | 
The Corporation has begun a comprehensive review of its consumer 
publications. The review will be completed in March 2003 and will look at the 
role of both electronic and print publications in consumer decision-making, 
timing of trip planning, and the type of information that is most in demand. 
Consumer market research will be conducted to determine how people are ~ 
using the publications. Secondary research will involve a competitive review of 
other tourism jurisdictions relative to consumer publications. 


The Corporation will assess the potential for advertising revenue from its 
French-language publications. If there is good potential, appropriate industry 
partners will be approached for advertising participation in the 2003 
publications. 


In conjunction with the publications review currently underway, the 
Corporation will review its inventory and distribution policies and revise them 
as appropriate. 











Festival and Event Grants 


The Corporation administers two grant programs: the Tourism Event Marketing 
Partnership (TEMP) and the Industry Partnership Proposal (IPP). These grant programs 
provide financial assistance to eligible organizations for their marketing and advertising 
initiatives. The programs’ primary objective is to encourage and increase tourism in the 
province. The amount of the grants they provide is based on the percentage of an 
organization’s eligible project costs. The maximum grant amounts per eligible recipient are 
$50,000 for the TEMP and $60,000 for the IPP. During the 2001/02 fiscal year, 
approximately 100 eligible recipients received grants totalling $2.7 million. 


In 2001, the Ministry engaged a consultant to compare the province's funding and other 
initiatives in support of festivals and events with those of other jurisdictions. The consultant 
made a number of recommendations, including the need to streamline application forms 
and develop a province-wide festival and events strategy. 


We reviewed the Corporation’s administration of its grant programs and found that grants 
were properly reviewed and approved according to program eligibility and funding criteria. 
However, improvements were required as follows: 


One of the key goals of the TEMP program is to develop off-season events. Applications 
for festivals and events that take place in the spring and winter (the off-seasons) should 
be given priority. The Corporation does not set a specific deadline date for grant 
applications but requires applications be submitted four months prior to the date of the 
event. We reviewed the 2001/02 approved applications and found that six months into 
the fiscal year (September 2001) the TEMP’s budget had been fully committed, 
primarily to summer and fall events. As a result, no budgeted funding was available for 
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winter and spring events that had not, as of September 2001, submitted their 
applications for review and approval. We were informed that the Corporation had to 
allocate additional funding to support off-season events. 


* One of the Corporation’s key performance measures for the grants programs is the value 
of contributions, both financial and in-kind, that festival and event organizers raise from 
other sponsors. The information on the value of sponsorship revenue is a measure used 
by the Corporation to demonstrate its effectiveness in developing partnerships. 


We noted that the information to support this performance measure needed to be more 
closely monitored. For instance, when applying for a grant, festival and event organizers 
are required to submit letters from each of their sponsors indicating what funds they 
have committed; however, we observed that only half of the applications sampled 
submitted sponsor letters. As well, the Corporation had no procedure for periodically 
verifying the accuracy of the sponsorship contributions and therefore could not be 
assured that the information it was using to measure its performance was accurate. 


* After festivals and events have taken place, grant recipients are required to submit a 
post-project report to the Corporation that details how the grant was spent and outlines 
the results obtained. However, the Corporation had not received the required reports 
from 30% of the recipients. The Corporation also did not assess whether the grant 
programs met the objective of increasing tourism in the province. 





To ensure that the financial assistance provided to festivals and events - : : 
through its two grant programs achieves the overall objective of encouraging © 
and increasing tourism in the province, the Ontario Tourism Marketing 


Partnership Corporation should: 


eligible festiva land event operators ; _ 


* develop a formal, province-wide strategy for providing financial support to. 
* more effectively monitor grant approvals to help achieve the goal of | 


developingoff-seasonevents; = st , 
¢ develop procedures to periodically verify the sponsorship commitments 
that are generated through festival and event organizers; and 
° ensure that grant recipients submit the required post-project reports and 
review all reports received to assess the success of the grant programs. 


Ministry and Corporation Response 


A study commissioned by the Ministry identified festivals and events as ke 4 
players in reflecting community culture, history, and traditions, thereby having 








tremendous value in drawing national and international visitors and reaping 
the economic benefits. : 








oo SSSSSSSSSSSFSSSSSSSssheseseseseFeFesSsSs 


Tourism Program 273, 


| VFM Section 3.10 | 








pee wWOULIUTE Ws TV | 

















The Ministry, in partnership with the industry, will publicly release the study’s 


recommendations in November of 2002. 


The Corporation has now developed and implemented major Ontario event 
funding criteria for large, world-class events and festivals in Ontario as part of 
its Tourism Event Marketing Partnership Program. 


The Corporation has recently negotiated an agreement with ministry field 
services staff to identify off-season and winter-season events for program 
support. Funds have been allocated to support events during these seasons. 


To support its oversight of sponsorship revenues, the Corporation will work 
with the Ministry’s Internal Audit Services to develop and implement 
procedures that will periodically verify these revenues. 


The Corporation will continue to require post-project reports from event 
organizers and ensure future funding is contingent on receiving such reports. 
For events not requesting additional funding, the Corporation will monitor to 


ensure the event was implemented as outlined in the proposal. 


The Corporation will develop a formal evaluation framework in which all post 
project reports will be analyzed to assess program success and will use this 
information to improve program decisions in the future. 








TOURISM OPERATIONS AND SERVICES 


Travel Information Centres 


The Ministry operates eleven year-round and seven seasonal travel information centres with 
annual expenditures of approximately $4.2 million. In addition to welcoming tourists and 
providing travellers with information on Ontario, the objectives of the centres are to 
encourage visitors to stay longer, return more often, and spend more money in Ontario. For 
the 2001 calendar year, the travel information centres received approximately 2.3 million 


visitors. This represents a decline of almost 25% in the number of visitors using a travel 
information centre since 1998. 


In 1997, the Ministry surveyed visitors to determine their opinion on the effectiveness of the 
travel information centres. The survey indicated that, as a result of the information obtained 
at a travel information centre: approximately 13% of the visitors extended their stay by an 


average of 1.7 days; 64% of the visitors intended to return to Ontario; and during the 


survey period (July to September 1997) tourists spent an estimated $12 million more in the 
province. 


In 1998, the Ministry’s Internal Audit Services reported on the operations of the travel 
information centres. In their report, Internal Audit Services concluded that the Ministry's 
survey was an adequate measurement tool and recommended that the survey be conducted 
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every two years. The Ministry agreed they would do so provided sufficient funding was 
available. 


The Ministry has not conducted a survey of visitors to travel information centres since the 
1997 survey and does not have other performance measures in place to measure the 
effectiveness of the centres in meeting their objectives. In 2001/02, the Ministry had 
planned to conduct a similar survey to obtain a better understanding of customer needs and 
to identify actions required to meet visitors’ expectations and increase the number of visitors 
to Ontario. However, at the time of our audit, the planned survey had not been 
undertaken. 





[ : 
Recommendation 


To help assess whether provincial travel information centres are effective in 
meeting their objectives, the Ministry should develop procedures to 
periodically evaluate whether the centres continue to encourage visitors to 
Stay longer, return more often, and spend more money in Ontario. 


Ministry Response 


The Ministry agrees that a thorough evaluation of the travel information 
centres to ensure objectives are being met is important for the future 
effectiveness of operations. 


The Ministry will continue to monitor customer satisfaction and the 
development of information services throughout travel centres. We will 
continue to monitor the effectiveness of the travel centres and their ability to 
deliver travel information services that entice visitors to stay longer, return 
more often, and spend more money in Ontario. 


The Ministry will also review opportunities that will enhance the centres’ ability 
to create a positive image of Ontario, through the development of a business 
case that provides a long-range plan to guide the future direction of its travel 
information centres and meets the ever-changing visitor expectations for 
easily accessible information and travel choices. 














Maintenance of Agency and Attraction Assets 


The Ministry's tourism agencies and attractions are responsible for maintaining a broad 
range of assets, including 150 historic heritage buildings. Heritage assets are usually 
considered irreplaceable and are intended to be preserved in trust for future generations. 
Most of the tourism agencies’ and attractions’ assets have had no significant capital 
investments for repair and maintenance in recent years. Tourism agencies and attractions 
require significant capital investment to remain competitive and attract repeat visitors. 
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Recommendation 


Se 


In its 2000/01 capital plan, the Ministry indicated that without multi-year capital 
investments for rehabilitation and improvement, some of the sites at its attractions might 
need to be closed due to the level of disrepair, which could result in reduced attendance 
and revenues. The government's capital planning process requires that the Ministry conduct 
1 formal assessment of the value and condition of its assets, including those of a historic or 
heritage nature. However, for most assets, no monetary value was assigned and no condition 
assessment done. In those instances where a condition was provided, it was often too 
imprecise to determine the amount of funding required to maintain the asset. For instance, 
some assessments identified an asset as “deteriorating” or fait 


Many of the assets at the province's agencies and attractions are considered by both ministry 
and agency management to be in serious need of repair. For the 2001/02 fiscal year, 
tourism agencies had estimated that $75 million in capital funding was required over the 
next five years for repairs and maintenance. The Ministry received approval for $30 million 
over the next five years. During the 2001/02 fiscal year, the Ministry allocated the $5 
million in funds received to high-priority health and safety capital projects. No funding was 


available for regular repairs and maintenance or for new capital projects. 





To ensure that its tourism agencies’ and attractions’ assets are adequately 
maintained for the benefit of future generations and provide a safe 
environment for staff and visitors, the Ministry should: 


e conduct a formal assessment of the value and condition of all of its assets, 
especially those of a historic or heritage nature; and 


e develop a long-term capital maintenance plan to identify the funding 
needed to rehabilitate the capital infrastructure and preserve assets. 


Ministry Response 


The Ministry concurs with the recommendation that a formal assessment of 
the value and condition of all its assets is key to a long-term maintenance 
plan—to identify funding needed to rehabilitate infrastructure and preserve 


assets and to build solid business cases for continued investment by the 
province in its assets. 


The Ministry currently does not have operating funds available and will need to 
examine alternative avenues to implement the recommendation related to the 
formal assessment. However, the Ministry will continue to prioritize health and 
safety needs among other capital requests on an annual basis. 


The Ministry is working on the development of a more strategic approach to 
capital maintenance planning. 
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Service Standards and 
Accommodation Rating Systems 


One of the primary objectives of the Ministry of Tourism and Recreation is to encourage 
and promote improvements in the standards of accommodations, facilities, and services 
offered to travellers in Ontario. 


Accommodation rating systems enable a future visitor to plan and choose accommodations 
based upon a quality-assurance system. With the increased use of the Internet, potential 
travellers can now plan their trips by accessing the Web sites of prospective destinations. To 
remain competitive, potential destinations must offer a variety of experiences and provide an 
indication of the quality of the service or attraction. 


Although private organizations have established rating systems for accommodations, 
Ontario is one of a few leading tourist destinations that does not have province-wide quality 
assurance standards or an accommodation rating system. Some Canadian provinces have 
province-wide, voluntary, accommodation rating systems that are either government 
operated or administered by the private sector. In 1999, as part of the Ministry’s draft 
Tourism Strategic Plan, the need for an industry-led rating system for hotels, motels, and 
campgrounds was highlighted as an important initiative to enhance industry 
competitiveness. 


We reviewed several recent attempts by the province to develop tourism service standards 
and accommodation rating systems and noted the following: 


* In March 2001, the Ministry agreed to provide $155,000 towards a tourism excellence 
program. The goal of the program was to raise the level of service and professionalism in 
Ontario's tourism sector to international standards. The program consists of three levels; 
commitment, implementation and results. As of March 2002, only 15 businesses had 
registered in the program and only five of these businesses had achieved the first of the 
three levels of excellence. If the program is going to meet its goal of improving the level 
and quality of service provided, more tourist establishments need to be registered. 


* In 2000, the Ministry of Agriculture, Food and Rural Affairs, through its Rural Jobs 
Strategy Fund, agreed to provide $1.1 million to two private-sector organizations to 
develop voluntary rating systems for roofed accommodations (hotels, motels, lodges, and 
resorts). However, as of March 2002, the private organizations had only rated 170 of 
the estimated 5,500 establishments in Ontario. In contrast, another Canadian province 
reported that more than 2,800 accommodation establishments were members of its 
rating system and they were targeting an annual increase of 2%. 


Due to a lack of support from the industry, the Ministry of Agriculture, Food and Rural 
Affairs withdrew funding from one of these organizations. It funded a third private 
organization to deliver a quality assurance program for campgrounds. Staff of the 
Ministry of Tourism and Recreation were not aware of the results achieved for this 
program. 
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Recommendation 


To help encourage and promote improvements in the standards of 

accommodations, facilities, and services offered to travellers in Ontario, the 

Ministry should: : 

° review accommodation rating systems in other provinces and those 
supported by other Ontario ministries; 


° take a lead role in encouraging and promoting the development of 
province-wide rating systems in Ontario; and 


e integrate any resulting rating systems into its information network. 


Ministry Response 


The Ministry will be undertaking a review in December 2002 of the Tourism 
Excellence Program, a voluntary program, to determine the reasons for the 
lack of participation by tourism operators. 


The Ministry will explore ways to renew the interest of tourism operators in the 
Tourism Excellence Program and implement changes required to ensure the 
program is accessible and of interest to tourism operators/businesses. 


The Ministry will explore with the Corporation the concept of rating systems 
and the value of incorporating rating systems into our programs. 


Currently, the tourism industry uses the Canada Select Program as its rating 
system, which is administered by a tourism industry alliance. 








Consumer Comments and Complaints 


To encourage and promote improvement in tourism establishments and services, the 
Ministry invites travellers to provide comments and complaints regarding their visits to 
Ontario. Comments and complaints are received primarily at travel information centres or 
by direct contact with other ministry offices. 


The Ministry of Tourism and Recreation Act does not give the Ministry the direct authority 
to collect personal information, including any personal information contained in a 
comment or complaint. When written correspondence is received, ministry staff are 
instructed to remove any personal information. After receiving the written authority of the 
complainant, the ministry forwards the original complaint to the public or private 
organization that can provide some assistance. 


The Tourism Act states that the Minister may appoint one of more persons to investigate, 
inquire into, and report to him or her upon any matter connected with or affecting the 
tourism industry. However, the 1998 amendments to the Regulations under this Act 
eliminated the Minister's broad investigative powers by indicating that the Act and 
Regulations apply only to tourist establishments that make use of Crown land. 
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Consequently, the Ministry has limited authority to investigate public comments and 
complaints arising from products and services delivered by most of the tourism industry. 


Consumer comments and complaints about goods, services, or travel experiences can 
provide useful information about the quality of the services provided by the Ministry and 
the tourism industry. However, the Ministry does not have a formal process in place to track 
the number and type of comments or complaints it receives each year. Ministry 
correspondence indicates that the majority of complaints it receives relates to 
accommodations, attractions, and border crossing, The tracking of comments and 
complaints could reveal trends over time and would support the Ministry’s activities to 
encourage and promote improvements in the standards of the province’s accommodations, 
facilities, and services. 











: Recommendation — 


To help. assess the Selly of: service ‘and consumer r satisfaction with Ontario’ Ss 
tourism experiences, the Ministry : should: 


< reassess the need for the current restrictions ¢ on n responding to concerns 
and complaints from the public; and— _ 


-¢ establish standard guidelines for Soran: consolidating, evaluating, and 
erg action on eae comments a connie. _ 


Ministry and Corporation F Response 


The Ministry agrees itis important to De. aware 2 of te. quality of its services and — 
consumers’ satisfaction with Ontario’s tourism experiences. The Ministry will 
review its consumer complaints DIQCESSES and will seek legal oe Lge on the 
legislative requirements. .=-—se 


The Corporation and its service ppravicer Have pod teaonihes the number and 
nature of complaints received via the Tourism Consumer Information System, 
which includes a 1-800 number, e-mail, web concierge, and regular mail. 
Reporting will become more accurate and formal as the system moves from 
“pilot” to “ongoing run” mode, which is expected to happen by the end of 2002. 
The feedback will be used to track consumer satisfaction with our services and 
operation of the services. We will make improvements and adjustments based 
on this feedback and other information throughout the life of the system. 














Management of Consulting Services 


The Ministry and the Corporation engage management consultants for a number of 
reasons, including the lack of in-house expertise, the need for independent review, and the 
need for resources over short time periods. During the three years ended March 31, 2002, 
the Ministry and the Corporation entered into more than 200 consulting contracts at a 
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value of $9 million. We selected a sample of contracts and observed that the Ministry's and 
the Corporation’s procedures did not ensure compliance with the Management Board of 
Cabinet requirements for engaging the services of a consultant. For example: 


© Most projects examined did not have an adequate business case on file, and terms of 
reference were either not prepared or did not provide sufficient detail on the project's 
scope and deliverables. In several instances, the contract terms and conditions had not 
been reviewed and approved by legal counsel to ensure the interests of the Ministry and 
the Corporation were appropriately protected. We also found that the performances of 
consultants had not been formally evaluated as required after projects were completed. 


¢ The required competitive acquisition processes often did not meet the approval and 
competitive acquisition requirements. Several contracts were awarded directly to the 
vendor without competition even though the ceiling price exceeded the tendering limit 
of $25,000. Other contracts with a ceiling price of less than $25,000 did not include, 
as required, anticipated expenses—which would have brought them over the $25,000 


tendering limit. 


© We observed situations where projects were split into separate contracts each valued at 
less than $25,000, thus avoiding open competition requirements. The consultants were 
hired continuously on year-to-year contracts. In some instances, the total value of all 
follow-on agreements was more than double the ceiling price of the initial agreement. 
These extensions were made without evaluating the consultant’s performance or 
assessing the reasons why the original deliverables had not been provided. 


¢ Inseveral instances, consulting services had commenced prior to the contract being 
signed or finalized, and in two instances, no contract was prepared at all. In another 
instance, a $40,000 contract extension was signed even though the original contract 
had expired a year earlier. During the period when the consultant was not under 
contract —between the expiration of the original contract and the signing of the 
contract extension—the consultant was paid over $25,000. 


We observed instances where the relationship between the Corporation and consultants 
could be construed as an employee-employer relationship and therefore may have been 
subject to a number of income tax rules, such as the requirement to withhold income 
tax and Canada Pension Plan contributions. For example, we noted that a consultant’s 
contract stipulated a daily per diem rate, but the consultant received a monthly payment 
regardless of the actual number of days worked. In addition, one consultant was using 


business cards from the Corporation and was giving his business address as the 
Corporation’s head office. 









Recommendation 


To help ensure that consultants are engaged in a fair and competitive manner 
and that value for money is being achieved, the Ministry and the Ontario 
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Tourism Marketing Partnership Corporation should comply with the | 
Management Board of Cabinet (MBC) directives on the acquisition of 
consulting services. Special emphasis should be placed on improving the 
monitoring and evaluation of consultants’ DACRE ee and on improving 
contracting practices. 











Ministry and Corporation Response oe 


The Ministry has implemented ; a Web-accessible Spending peedscn 
Framework that clarifies the requirements set out in the MBC directives on 
consulting services. In addition, a “Best Practices for Managing Consulting — 
Services” pamphlet that emphasizes the importance of evaluating consulting — 
performance will be provided to all managers. Presentations will be made to — 
managers to ensure the Framework ae Best t Practices are understood and 
adhered to. _ -r—“—itOCOwCONWCOUCsisS«atSOiSCSCdésésCCCW _ 


The Corporation v will pur in controls iB bedinniig Sapieinbar 2t 2002 to Improve 
procurement practices and execution of consulting contracts to ensure they 

) comply with the MBC directives. The Corporation will consult with 

Management Board Secretariat and Shared Services Bureau on procurement 
when hiring consultants and consult performance evaluation tools to support — 

contract management. Training on procurement and contract management for 
managers and Staff i is planned for fall 2002. T. he objective of the trainingisto _ 

_increase managers’ ‘understanding of contract management, and i improve their 
skills i in monitoring and evaluating the work of consultants and in providing 

_ required documentation for consultants’ EP eenre results, cee and 

. contract conclusions. == _. 


| VFM Section 3.10 | 














INFORMATION MANAGEMENT 


Tourism Consumer Information System 


In 1999, the Ministry and the Corporation decided to develop a new Tourism Consumer 
Information System (System) to promote Ontario as a tourist destination using an integrated 
telephone call centre and Internet system. In 2000, the Ministry obtained approval from 
the Management Board of Cabinet to engage a private-sector company to develop and 
operate the new system. The Management Board of Cabinet indicated that the entire 
project's costs over four years were not to exceed $17.7 million. 


In August 2000, the Corporation issued a request for proposals (RFP), prepared by a 
consultant, for the development of the proposed System. In January 2001, the successful 
supplier was selected and the Corporation began negotiating a contract with the help of a 
consultant. At that time, it was anticipated that the proposed system would be operational 
by the summer of 2001. However, a contract was not finalized with the successful vendor 
until August 2001. The contract that was arrived at after lengthy negotiations was for a 
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three-year period and was worth $16.2 million. At the time of our audit, the Corporation 
anticipated that the new system would be operational by June 2002. 


We examined the processes and procedures followed by the Ministry and the Corporation 
in the approval, acquisition, and development of the System. We found that the Ministry 
followed the Management Board of Cabinet directive for competitive tendering . However, 
we noted a number of areas where the Ministry and the Corporation needed to improve 
their information technology project management. Specifically, we noted the following: 


© Neither the Ministry nor the Corporation had the necessary in-house staff skills to 
develop the RFPs, negotiate the contract, and manage a project of this magnitude and 
complexity. As a result, corporation and ministry staff responsible for the project relied 
on staff from other ministries and private consultants to provide expert advice and 
support during negotiations and all phases of the project. Although such specialized 
expertise was necessary, we found little or no evidence of a transfer of knowledge from 
consultants to ministry staff to avoid a continuous reliance on consultants. The 
experience gained can facilitate the monitoring of the system once it is operational and 
help in the retendering and renegotiation of the existing contract when it expires. 


¢ Akey component of the development of the System, as outlined in the RFP, was a 
$2-million marketing strategy. However, we noted that the intention to fund the 
marketing strategy as part of the development of the System was not disclosed in the 
Ministry's submission to the Management Board of Cabinet for approval. In addition, 
the Corporation did not receive specific approval for this marketing strategy from the 
Advertising Review Board, which is responsible for approving all government 
advertising or communication projects in excess of $500,000. 


© The REP developed by the consultant did not provide prospective bidders with 
sufficient information on the expected system requirements, the scope of the work, 
service standards, or remedies for nonperformance. Consequently, prospective bidders 
raised over 100 supplementary questions, which had to be clarified during the contract 
negotiation process. The lack of clarity in the RFP was a major reason for the delay in 
the tendering process and contract negotiations. 


In the RFP, the Corporation indicated its desire to have the System developed and 
operated on a fixed-price basis. For a fixed-price contract, the risks of cost increases are 
borne by the successful bidder unless there are significant changes to the original 
specifications. The financial terms and conditions agreed to with the vendor were based 
ona fixed price for the development and maintenance of the system. However, the 
supplier does not assume any of the financial risks associated with fluctuations in service 
levels but the Corporation does. For example, the supplier is guaranteed a minimum of 
$950,000 annually regardless of the actual volume of telephone calls and Internet 
activity; but should the number of calls exceed the estimate, then the Corporation 
would be required to pay an additional $4 per contact. 
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Recommendation 





To support the efficient and economic administration of systems development 
projects, the Ministry and the Ontario Tourism Marketing Partnership 
Corporation should ensure that: 





° atransfer of knowledge occurs from consultants to staff to avoid a 
continuous reliance on consultants; 


e all significant deliverables and options are included in Prolect business 
cases and all required approvals have been received; 


e requests for proposals are well researched and provide a clear description 
of project requirements; and 


¢ financial and operational risks are adequately Hae e and shared with the 
vendor. _ 


Ministry and Corporation Response 


The Corporation will facilitate a transfer knowledge oni consultants to staff 
_ to the extent feasible since consultants’ work is often highly specialized. We 
will continue to apply knowledge and skills to future large-scale projects to 
ensure that project teams sufficiently research the business case, consider the 
_ options, and seek approval for the project plan prior to issuing a request for 
proposal (RFP). The Corporation will continue to consult with Shared Services 
‘Bureau and the Ministry’s Internal Audit Services as appropriate to ensure 
_RFP’s are clear and that risks are e identified, eget, and Zeer repretely 
shared with the vendor. _ _ 


| The Corporation i is ourere working v with te Ministry’ Ss internal Audit 

| Services ona plan to educate the new Tourism Consumer Information System 
| teamon the details of the contract with its service provider. As well, training 
will be provided « on researching, writing, and managing RFPs. 





Internal Audit Services will conduct. a risk assessment on the Tougen 
Consumer Information System project. Based on this Meee risk pi noaton 
| strategies will be developed as appropriate. 








Facilities and Attractions Databases 


In 1998, the Ministry developed a tourism information database to support tourism 
industry planning and decision-making and to assist visitors at the Miunistry’s Travel 
Information Centres. The database contains information on accommodation 
establishments, tourist attractions, and golf courses. In 1999, selected information was 
transferred to the Corporation so it could also provide a tourist establishment database for 
public use on its Internet site. 


ee ee ee ee ee 
Tourism Program 283 





VFM Section 3.1 o| 





| VEM Section 3.10 | 


a 


In reviewing the administration and maintenance of the two databases, we observed a lack 
of co-ordination between the Ministry and the Corporation, as follows: 


© The Ministry and the Corporation were using different methods to verify the accuracy 
and completeness of the tourist establishment information contained in their respective 
databases. The Ministry annually engages consultants to survey all existing 
establishments to ensure its information is current and accurate. In the 2001/02 fiscal 
year, the cost of conducting the survey to update the Ministry's database was $129,000. 
To keep its database up to date, the Corporation encourages tourist operators to input 
their updated information on Its database. 


© The Corporation and the Ministry do not have a process that would periodically 
compare the information contained in their respective systems and update them as 
needed. Currently, the Ministry’s database contains approximately 10,000 
establishments, while the Corporation estimates it has 8,000 in its database. 


© The Ministry is actively searching for new tourist establishments to add to its database by 
acquiring non-tourism related databases, such as the databases for the Alcohol and 
Gaming Commission and for municipal property tax assessments. However, we noted 
that neither the Ministry nor the Corporation had entered into partnership agreements 
with the various regional and municipal tourist associations to share information about 
tourist establishments. 





Recommendation 


To help ensure that tourist information is collected in an economical and 
efficient manner and provides prospective tourists with complete and accurate 
information, the Ministry and the Ontario Tourism Marketing Partnership 
Corporation should: 


e establish procedures to share information between their current databases 
and consider establishing one shared database; 


° review the feasibility of developing a single method of obtaining and 
verifying data from tourist operators; and 


° determine the practicality of entering into partnerships to share data on 
tourist establishments with other organizations, such as municipal and 
regional travel organizations. 


Ministry and Corporation Response 


The Ministry will work with the Corporation to establish procedures and 
guidelines for sharing database information in 2003. 


The Ministry and the Corporation will in 2003/04 jointly review the feasibility of 


developing a common process for obtaining and verifying data from tourist 
operators. 
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The Ministry has agreements with the Municipal Property Assessment 
Corporation (a Crown agency), the Alcohol and Gaming Commission, and the 
Ministry of Natural Resources to exchange or buy data relevant to tourism. The 
Ministry also receives data from a number of municipalities around the 
province. 





The Ministry previously shared data with tourism associations; however, it 
discontinued this in 1999/2000 because the Ministry collects data that is not to 
be used for solicitation or sold. We are exploring ways to make data available 
for use by tourism stakeholders. = - 
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MINISTRY OF TRAINING, COLLEGES AND 
UNIVERSITIES 


3.11-Training Division 


eee 


BACKGROUND 


The mandate of the Training Division (Division) of the Ministry of Training, Colleges and 
Universities is to set standards for employment services and adult literacy, to help employers 
develop a skilled workforce to stay competitive, and to provide leadership on labour market 
and training issues. The Division’s programs and services are intended to assist both 
individuals and employers in increasing skill levels and to help individuals make the 
transition from unemployment to employment and from education and training to the 
labour force. 


Division expenditures for the 2001/02 fiscal year totalled $346.3 million and were 
distributed across four major areas, as outlined in the following illustration. 


Division Expenditures by Major Program for the 2001/02 Fiscal Year 


Job Connect 
$94.9 million and 
Summer Jobs Service 
$20 million 1 
(33%) 






Apprenticeship 
$80.9 million (23%) 


Literacy and 
Basic Skills Other programs and 


$66.2 million (19%) Sa million? 
0 


For the Summer Jobs Service, the Ministry was allocated $24.6 million, but approximately $4.4 
million was then transferred to other ministries for their own summer job programs. 


About $26 million of this $80.9 million was recovered from the federal government. 


Included funding of $48 million for TVOntario and federal funding of $10.6 million for community 
colleges, which flowed through the Ministry but was administered by the federal government. 


Source of data: Ministry of Training, Colleges and Universities 
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Colleges of Applied Arts and Technology (community colleges), school boards, and 
community-based, not-for-profit organizations form the network of agencies responsible for 
delivering three of the major transfer-payment programs: Job Connect, Summer Jobs 
Service, and Literacy and Basic Skills. Employers are the primary deliverers of 
apprenticeship training, while community colleges and private training institutions that are 
funded by the Ministry and the federal government deliver in-school training assistance. 


OBJECTIVES AND SCOPE 


The objectives of our audit were to assess whether the Ministry had adequate systems and 
procedures in place to: 


* measure and report on the effectiveness of the Division’s major programs in meeting 
stated objectives; 


* ensure that the Ministry and its delivery agencies are meeting labour market and 
participant needs in an economic and efficient matter; and 


* ensure key activities and programs are delivered in accordance with legislative and policy 
requirements. 


Our audit was performed in accordance with the standards for assurance engagements, 
encompassing value for money and compliance, established by the Canadian Institute of 
Chartered Accountants, and accordingly included such tests and other procedures as we 
considered necessary in the circumstances. 


The audit focused on the following major programs: Job Connect, Summer Jobs Service, 
Apprenticeship, and Literacy and Basic Skills. These programs account for approximately 
75% of the Division’s expenditures. The scope of our audit work included reviews and 
analyses of relevant ministry files, administrative policies, and procedures, as well as 
interviews with staff at both the head office and field offices. We also visited delivery agencies 
for each program to examine files and other supporting documentation and to interview 
staff. 


Prior to the commencement of the audit, we identified the audit criteria that we would use 
to conclude on our audit objectives. These were reviewed with and agreed to by senior 
management for the Division. 


We conducted our audit work during the period October 2001 to April 2002, with an 
emphasis on activities and expenditures in the 2000/01 and the 2001/02 fiscal years. 


The Ministry’s Internal Audit Services Branch had recently commenced an audit of controls 
over the administration of apprenticeship examinations and revenue collection that it 
expected to complete by summer 2002. Accordingly, we reduced our audit work in these 
areas. The Branch had not completed any audits that allowed us to further reduce the 
extent of our work in the programs we examined. 
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OVERALL AUDIT CONCLUSIONS 


The Ministry has taken steps to design systems for providing meaningful performance 
‘nformation about its major training programs and the delivery agencies it funds. At the 
time of our audit, the systems and processes necessary to ensure that services are delivered 
economically and efficiently and that the programs’ objectives and expected outcomes were 
being achieved had not been completed. Successful implementation of both the initiatives 
currently under way and our recommendations is essential to ensure that the needed 
systems and processes are in place. 


We also found that the Ministry did not adequately control the acquisition and 
management of consulting and other services acquired through agencies on the Ministry's 
behalf over the last several years. Our program-specific observations and conclusions are as 
follows. 


Job Connect and Summer Jobs Service Programs 


° While the Ministry had set clear expectations for the performances of its delivery 
agencies and linked funding to the achievement of those expectations, it did not have 
adequate procedures to ensure that the actual results that the agencies reported were 
reliable and that service-delivery requirements were being met. The Ministry had begun 
to collect and report participant and employer satisfaction and service co-ordination 
information. Once adequate controls over data reliability are established, this 
information will help it to determine whether agencies are meeting participant and 
labour market needs efficiently. 


° The delivery agencies we visited for these programs were complying with program 


requirements, including appropriate assessments of participant needs and maintenance 
of required documentation. 


Apprenticeship Program 


The Ministry was not monitoring the quality of apprenticeship training provided by 


employers and in-school training providers. 


Efforts to co-ordinate enforcement responsibilities and share information with the 
Ministry of Labour and other bodies responsible for workplace inspections have not 
been sufficient to determine the extent to which uncertified individuals are working in 
restricted trades. Effective enforcement of restricted trades is necessary to ensure 


legislated objectives for protecting public and workplace safety are met and to maintain 
the value of obtaining certification in restricted trades. 


Training standards, curricula, and examinations had been updated for most trades with 
significant apprenticeship enrolment but a few were decades old. 


The Ministry was developing an apprenticeship information system that it expected to 
have in place by 2004. Until then, it will not have the performance information needed 
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to link funding for in-school training providers to their performances in providing 
quality training or to report publicly on program outcomes. 


Literacy and Basic Skills Program 


* The Ministry had not linked funding to the actual activity levels and performance 
achieved by its delivery agencies. The systems and procedures that are needed to collect 
and report meaningful performance information were under development. 


* The procedures established by the Ministry to monitor whether delivery agencies were 
meeting expectations were not sufficiently focused to permit effective oversight of 
agency performance and consistency of follow-up on corrective action. 


¢ The delivery agencies we visited were complying with program requirements. 
Acquisition and Management of Consulting and Other Services 


We noted that since the 1998/99 fiscal year the Ministry has paid two third-party, not-for- 
profit agencies funds totalling approximately $11 million to undertake significant projects 
and to purchase related consulting, information technology, and other services for the 
administration of the Job Connect and Literacy and Basic Skills programs. In doing so, the 
Ministry did not adhere to prudent purchasing practices and did not obtain the approvals 
from the Minister and Management Board of Cabinet that would have been required if the 
projects had been undertaken and the services acquired by the Ministry directly, which 
resulted in the following: 


* services amounting to about $8 million were acquired from private-sector suppliers with 
little or no competition; a further $2 million in services were provided directly by one of 
the not-for-profit agencies; 


* information systems project requirements were not well defined before the projects 
began; 

* funds were advanced before they were required in some cases; 

* contract monitoring was not adequate to ensure all billings were appropriate; 


* there was no documentation to support a significant increase in project administration 
fees; and 


2 GST charces totalling $600,000 were incurred because the agencies were not GST- 
exempt—$235,000 of that amount was overbilled and should be recovered. 


While the Ministry’s Job Connect information system has allowed it to improve the 
administration of the program, it could not demonstrate that it obtained value for money in 
developing the system or in undertaking the other projects we examined. The Ministry has 
committed to improving and has already taken some action to strengthen its procedures for 
managing all existing and future projects. 
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DETAILED AUDIT OBSERVATIONS 


JOB CONNECT AND SUMMER JOBS SERVICE 
PROGRAMS 


Job Connect primarily assists unemployed and out-of-school youth between 16 and 24 
years of age in gaining the skills and knowledge necessary to find and keep employment. It is 
delivered at 123 sites across Ontario by 78 agencies consisting of 22 community colleges 
and 56 not-for-profit organizations. The agencies provide the services outlined in the 
following table. 


Job Connect—Services and Participants, 2000/01 


Number of 
Participants 
Receiving | 
Service 


Service Delivered 





Information and referral service: a walk-in, self-service community employment 
resource centre providing information on careers and occupations, the local labour 
market, training opportunities, job search strategies, and employment related 
workshops. 


35,840 





Employment planning and preparation: a service that assists participants who require 
more structured assistance to plan and conduct job searches (includes instruction in 59,635 
resumé preparation and effective interview skills). 





Job development placement supports: a resource for participants receiving 

employment planning and preparation services but who require more intensive 

assistance to develop their skills and gain experience through such supports as job 26,725 
trials, volunteer placements, subsidized on-the-job training opportunities, and direct 

employment placements. 





Subsidized placements: placements that vary in duration but cannot exceed a period of 
six months, with a yearly maximum subsidy of $ 4,000 per participant. Both duration 
and subsidy levels are negotiated based on the training needs of the individual, the 
employer's willingness to provide the training, and the placement support services that 
will be provided by the delivery agency. The hourly subsidy cannot exceed the 
minimum wage. 


22,258 
Source of data: Ministry of Training, Colleges and Universities 


About 97% of the almost $95 million of program expenditures in 2001/02 were transfer 
payments to agencies. Of the $91.7 million provided to these agencies, 65% was for service 
delivery, 30% was for training subsidies, and 5% was for special projects. 


Through the Summer Jobs Service, employers can receive a $2-per-hour training subsidy to 
hire students for up to 16 weeks from April to September. This program is delivered 
primarily by the same agencies that deliver the Job Connect program. For the 2001/02 
fiscal year, about 27,000 students were expected to receive summer employment services. 


Of the $20 million provided to delivery agencies in 2001/02, about 80% was for training 
subsidies and the remainder was for program delivery and administration. 
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Measuring and Reporting on Program 
Effectiveness 


We concluded that the Ministry has made substantial progress in ensuring that funding 
provided under the Job Connect program complies with the Management Board of 
Cabinet Directive on Transfer Payment Accountability. Specifically, the Ministry had 
developed clear expectations and performance measures that focused on measurable results 
rather than on process to assess the achievement of program objectives and was adjusting 
funding based on the performance of transfer-payment recipients. 


The Ministry began implementation of a Continuous Improvement Performance 
Management System (CIPMS) for evaluating the performances of Job Connect delivery 
agencies on April 1, 2000. As part of this management system, the Ministry has established 
six core performance measures that relate to employment planning and preparation services 
and job development placement supports. The following table illustrates the core 
performance measures. 


Core Measures and Service Delivery Standards, 2001/02 


Customer service Participant satisfaction Questionnaires completed by agency at 
30% program exit; 85% satisfaction level set 
as an interim target. 





Employer satisfaction 


Effectiveness 50% Participant employed Standard is set at 64% for paid 
three months after exiting | employment based on 100% survey. 
the program On a sample basis, agencies conduct 
6- and 12-month follow-ups, but those 
results are not counted as a core 
measure and no standard has been set. 


A 





Participant profile Composite indicator derived from the 
sum of nine factors that indicate level of 
need for services. 


Efficiency 20% Service co-ordination Composite indicator derived from a 
number of external linkages at exit that 
include participants registered in English 
as a second language, literacy training, 
federal training, and returned to 
education. 





6* Intake Actual number of participants receiving 
service compared to planned. Standard 
set at 100%. 





These three measures form part of a provincial service delivery standard that delivery agencies must 
meet to continue to qualify for funding. The remaining three core measures have interim targets that 
the Ministry intends to refine and include as part of the provincial standard once it is satisfied with the 
data being collected and reported. Until then, agencies are still expected to achieve their interim 
targets. 


Source of data: Ministry of Training, Colleges and Universities 
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Agencies that fail to meet the provincial service delivery standard are required to develop 
plans for improvement within clearly defined timelines. If an agency fails to meet the 
targeted timelines, the Ministry may terminate funding. 


The Ministry was using information reported by its delivery agencies to shift funding to 
those agencies that meet performance targets and demonstrate the ability to increase their 
‘atake numbers. Ministry consultants have worked closely with underperforming agencies 
to help them achieve improvement targets and deadlines. The Ministry has also used 
independent coaches to help agencies improve service delivery and results. 


The CIPMS isa significant initiative that can assist the Ministry to manage the program 
effectively and to demonstrate that it has done so. However, given the funding consequences 
if the provincial service delivery standard is not met, there is a significant risk that delivery 
agencies will simply report the expected results rather than actual performance. 

Accordingly, the Ministry needs to take steps to verify that the performance information it is 
collecting is reliable and that the performance targets it establishes are appropriate. In this 
regard we noted the following: 


* The Ministry relies on agency personnel to submit performance information. Agency 
personnel complete participant and employer satisfaction questionnaires and conduct 
three-month employment follow-ups, primarily over the telephone, without third-party 


verification. 


© The achievement targets for the core measures that make up the provincial service 
standard have not been updated since their introduction in 1999/2000. Since 
implementation of the service standard, delivery agencies reported continually improved 
results. As of December 2001, agencies overall exceeded the provincial standard by 
11% and only 2 of 78 agencies reported that they had not met it compared to 15 in 
2000/01. Agencies also reported that, on average, 70% of their participants were 
employed three months after exiting the program versus the provincial standard of 


64%. 


As part of the CIPMS, the Ministry plans to develop and implement a benchmarking 
process that would allow the comparison of the program performances of Job Connect 
to that of similar programs in other provinces and jurisdictions. At the time of our audit, 


the process had not been developed. 


For the Summer Jobs Service program, the only performance measure established was a 
comparison of actual to planned intake targets for subsidized placements. Intake 
information is not sufficient to assess the effectiveness of the Summer Jobs Service program 
or its delivery agencies. More useful performance information would include participant 
and employer views of the need for the program and the level of satisfaction with both the 


delivery agency and the program, as well as the extent of any ummet demand for 
placements from youth or employers. 
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[ Recommendation 


To help ensure its Continuous Improvement Performance Management System 
(CIPMS) operates as intended to monitor and improve the overall performance 

of delivery agencies for the Job Connect and Summer Jobs Service programs, 

the Ministry should: 


e establish procedures to periodically verify the reliability of the performance 
information reported by delivery agencies; 


° complete the development of benchmarking for the Job Connect program 
to allow comparisons with other jurisdictions; and 


¢ establish more meaningful measures for assessing the performances of the 
Summer Jobs Service delivery agencies and the effectiveness of the 
program overall. 


Ministry Response 


We agree with this recommendation. Currently, the 287 statistics reported by 
agencies to the Ministry are validated through cross-references in the 
Ministry’s database system. In addition, agency follow-up is conducted where 
system checks show anomalous data. We are pleased this audit did not find 
any inconsistencies among information in the client files, the agency database, 
and the Ministry’s database. In order to improve the process in 2002/03, the 
Ministry is updating and documenting site visit procedures to include a 
process for verifying information, with a particular focus on the recently 
implemented service co-ordination and customer satisfaction ratings. 


The benchmarking process will be started in 2003/04, as per our CIPMS 
implementation work plan. _ _ 


We anticipate that all performance measures and standards will be fully 
implemented in the 2003/04 fiscal year, and, with information from that year, we 
plan to benchmark the results. In the interim, in 2002/03 the Ministry’s Award 
for Excellence and the Innovations Awards provide an opportunity to validate 
and examine the results of high-performing agencies to test our internal 
benchmarking process. In addition, because agencies are required to commit 
to targets that exceed their prior-year results, program performance continues 
to improve even though benchmarks have not yet been identified and 
provincial standards have not been adjusted. 





The Ministry will factor in the need for more meaningful measures in the 





| overall evaluation of the Summer Jobs Service that is underway in 2002/03. 





Monitoring Compliance with Program Guidelines 


The Ministry revised the Job Connect and Summer Jobs Service program guidelines in June 
of 2001. The new guidelines describe the objectives and policies governing the programs 
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and indicate the documentation and verification required for participants’ files. We visited a 
number of delivery agencies, reviewed samples of their participant files, and concluded that 
they were complying with ministry guidelines. For example, we found that files were well 
documented and included all key information needed to determine participants’ needs and 
the services provided. Survey questionnaires were completed, and, where applicable, claims 
for training subsidies were properly determined. 


The program has 12 field consultants who act primarily as advisors to agencies. They review 
their assigned agencies’ annual budgets, business plans, and performance reports. Flowever, 
the annual visits that field consultants make to agencies do not check whether agency 
procedures and files meet ministry expectations or whether the reported performance 

‘ formation is accurate. To be cost effective, field consultants could use their knowledge of 
the agencies to assess the risk that an agency may fail to comply with requirements or may 
report inaccurate results. This assessment of risk would help consultants plan the frequency 
of visits and the extent of the review to be done at each agency. 





| Recommendation 


To help ensure that delivery agencies for Job Connect and Summer Jobs 
Service programs comply with ministry guidelines and that the performance 
information on which funding is based is reliable, the Ministry should establish 
a risk-based program of periodic visits by field consultants to delivery 
agencies. 


Ministry Response 


We agree with this recommendation. Agencies below performance standards 
are now visited by ministry staff for an in-depth assessment of risk, of 
compliance with program guidelines, and to establish ministry-directed 
performance improvement targets. A comprehensive review process is 
undertaken for all sites with ministry-directed improvement targets. 


In addition, other indicators, such as ongoing difficulty in data or financial 
reporting or customer complaints, generate agency visits/reviews. As noted, 
the audit field visits confirmed compliance with program guidelines and 
directives. 


All sites will be visited at least once within the three-year life of the funding 
contract. 








APPRENTICESHIP PROGRAM 


The Apprenticeship program is governed by two acts. The Trades Qualification and 
Apprenticeship Act (1990) governs thirty-four construction sector trades, and the 
Apprenticeship and Certification Act, 1998, proclaimed January 1, 2000, governs trades in 
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the industrial/ manufacturing, motive power, and service sectors. Each act establishes specific 
requirements for apprenticeship completion and the roles of the Ministry and industry. 
Each act also stipulates that certain trades may be practised only by individuals who are 
registered apprentices or who hold a certificate of qualification. Certification is optional for 


all other trades. 


Apprenticeship is a work-based training model that combines on-the-job training 
(approximately 90%) with classroom training (approximately 10%). The length of an 
apprenticeship can range from two to five years, during which time the apprentice must 
typically complete at least three in-school training sessions. 


Each apprentice signs a training contract with an employer that requires the employer to 
help the apprentice acquire both work experience and trade-specific competencies. Once a 
training contract is signed and filed with the Ministry, the apprentice is officially registered. 
Upon successful completion of the apprenticeship requirements, an individual receives a 
certificate of apprenticeship. For certain trades, the apprentice must also pass a trade-specific 
examination to obtain a certificate of qualification. Individuals not registered as apprentices 
are also entitled to take the trade examination if, for example, they can demonstrate that 
they have obtained the required work experience in another jurisdiction. As of January 
2002, there were 136 trades involving about 52,000 registered apprentices. Certification is 
mandatory for 20 of these trades. 


Program expenditures for the 2001/02 fiscal year included $58 million paid to training 
delivery agencies for in-school training and for updating programs (about $26 million of 
which is recovered from the federal government) and $22.9 million for the Ministry’s direct 
operating expenditures. 


The program is administered by some 250 ministry staff, about 100 of which are training 
consultants located in 26 ministry field offices across the province. These training 
consultants register apprentices and consult with employers and training delivery agencies. 


Measuring Program Effectiveness 


In June 1998, the Minister of Training, Colleges and Universities announced a long-term 
commitment to significantly increase the number of annual apprenticeship registrations to 
22,000 from the 1998/99 level of 11,000 to meet the demand for skilled workers and to 
support job creation. In its annual Business Plan, the Ministry reports the number of 
apprenticeship registrations but does not publish any other meaningful performance 
information about the program. Internally, it established two additional program 
commitments for which progress is to be tracked: increasing the number of active and new 
employers from 18,100 in 1999/2000 to over 23,000 in 2001/02 and increasing the 
proportion of apprentices who successfully complete their classroom training from 60% in 
2001/02 to 75% over the long term. 
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While tracking progress against these commitments will help the Ministry assess the degree 
of success in encouraging greater acceptance of and support for apprenticeship training, it 
will not be sufficient to assess the program’s contribution to meeting the demand for skilled 
workers in Ontario, a stated objective and priority for the program. Increasing the number 
of registered apprentices and active employers will not meet the demand for skilled workers 
unless apprentices complete their programs and acquire the training and skill sets needed by 
the labour market. 


Consequently, the Ministry needs information on apprenticeship completion and 
employment rates in relation to labour market demand. This, combined with participant 
survey information, would provide a better indication of the appropriateness of the training 
being provided and of any areas requiring improvement. For example, each year the 
Province of Alberta publishes the percentage of qualified apprentices obtaining 
employment, the percentage of apprenticeship graduates satisfied with their work 
experience and technical training, and the percentage of employers satisfied with the 
apprenticeship training system. 


In early 2002, the Ministry began developing a new information system for the 
Apprenticeship program; its targeted completion date is mid-2003. The Ministry expects 
that this new system will enable it to collect information to better manage the program and 
report on results. The Ministry has also begun a project to develop and implement 


meaningful, outcome-based performance measures that it hopes to have in place by 
January 1, 2004. 





Recommendation 
The Ministry should ensure that the information and performance management 
systems it is developing will, as soon as possible, allow it to begin reporting 
publicly on achievements with respect to apprenticeship completion and 
employment rates as well as the extent to which the apprenticeship program is 
meeting the expectations of apprentices and employers. 


Ministry Response 


We agree with the recommendation. The business planning and allocation 
process for 2002/03 includes a plan for the development of a continuous- 
improvement performance-management system for apprenticeship in Ontario. 
The Ministry is on target to implement outcome-based performance measures 
by January 2004 and thereafter to report publicly on achievements, including 
apprenticeship completion and employment rates. 
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Increasing Opportunities for Apprenticeships 


The Ministry was in the process of implementing several strategies to expand the 
Apprenticeship program to provide the skilled workers needed by the labour market. A 
new policy on developing apprenticeship occupations that responds to the demands for 
skilled labour remained to be finalized at the time of our audit. 


The Ministry added 29 new apprenticeship trades over the past four years although they 
only accounted for about 6% of total apprenticeship registrations in each of those years. In 
fall 2001, the Ministry identified another 14 occupations as potential apprenticeship trades, 
five of which had been delayed to fall 2002. A common problem in many jurisdictions is 
the difficulty of expanding the apprenticeship system beyond traditional trades, such as 
those in construction and the automotive sectors, into less traditional and faster growing 
occupations, such as those in business and commerce, health sciences, natural sciences, and 
social sciences. Many of the new trades introduced in Ontario over the past two to three 
years, and even trades under development, are in traditional areas of apprenticeship. 
However, if the Ministry is to meet its targets for the expansion of apprenticeship training, it 
will need to develop more apprenticeship programs in new occupations and sectors. 


We will follow up on the Ministry’s efforts to expand opportunities for apprenticeship in 
Ontario in conjunction with the follow-up of our recommendations in two years’ time. 


Updating Apprenticeship Standards 


To maintain the quality and credibility of the Apprenticeship program, apprentices and 
applicants for certificates of qualification must be trained and tested using up-to-date 
marketplace requirements. 


The Director of Apprenticeship has the statutory authority to approve apprenticeship 
programs, including applicable training standards, curricula, and examinations, as well as 
the responsibility for maintaining them. Apprenticeship training is intended to be industry 
designed, responsive to the needs of employers for highly skilled workers, and capable of 
expanding into new trades and areas of economic growth. Accordingly, under both the 
Trades Qualification and Apprenticeship and the Apprenticeship and Certification acts, the 
Minister may establish advisory committees made up equally of employers and employees 
for any trade or group of trades to develop and update training standards, curricula, and 
examinations. 


We reviewed the current status of training standards, curricula, and examinations for all 
restricted and compulsory trades as well as for voluntary trades (those where certification is 
optional) that had at least 50 or more registered apprentices. We found that the training 
standards, curricula, and examinations for most trades with significant apprenticeship 
registrations had been updated within the last three years, in accordance with ministry 
practice for reviewing standards. However, we noted that for several trades, both restricted 
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and voluntary, either the training standards and/or the examinations had not been updated, 
as indicated in the following table. 


Summary of Active Trades for Which Training Standards 
and Examinations Have Not Been Updated 


Applicants for 
Certificate of 
felTEV iitersiicela 

Over Past Five 


Apprentice 
Year Registrations 
Issued Over Past Five 
Years 


Training standard 
Motorcycle mechanic 





Domestic and rural electrician 
Examination 
Domestic and rural electrician 





Alignment and brakes 





Transmission technician 





Tower crane operator 





Mobile crane operator-2 





Construction millwright * 





Motive power machinist * 





*Voluntary trades 


Source of data: Ministry of Training, Colleges and Universities 
Prepared by the Office of the Provincial Auditor 


Several of these trades are restricted trades, and these trades represent about 30% of the 
trades with significant registrations for which the Ministry has the direct responsibility to 
keep the training standards and examinations up to date. Although these trades accounted 
for barely 2¥%—that is, about 1,000—of the 52,000 apprentices registered at 

December 31, 2001, it is our view that these are active trades that may well be more active if 
the training standards and examinations were up to date. While we found, as mentioned 
above, that the training standards and examinations for most trades with significant 
apprenticeship registrations had been updated within the last three years, the standards and 
examinations for the trades summarized in the table need to be updated as soon as possible. 
This would help ensure that apprentices and certificate-of-qualification applicants in these 
trades acquire the skills and knowledge they need and increase the value and attractiveness 
of these trades to those considering apprenticeship training in them. 
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Recommendation 


To help ensure that apprenticeship graduates acquire the skills needed to meet 
employer needs, the Ministry should ensure that all training standards and 
examinations are up to date and reflect current demands of the workplace as 
soon as possible. _. 


Ministry Response : | _ : 
The Ministry will take action to ensure that by December 2003 all training 


standards and examinations are updated as appropriate to meet the 
requirements ofindustry. = | _ 








Monitoring Program Quality and Compliance 


Because employers are responsible for 90% of apprenticeship training and in-class training 
providers are responsible for the remaining 10%, monitoring the quality of training 
provided by both parties is critical to the program’s success. Closer monitoring may also 
increase the likelihood that apprentices will complete their programs and obtain 
certification. However, at the time our audit, the Ministry had not developed either a policy 
on monitoring workplace training or procedures to monitor the quality of in-school 
training. 


Training consultants at the field offices we visited conducted few if any monitoring visits to 
employers and in-class training providers to determine compliance with the training 
contracts and service agreements, even though the legislation permits such inspections. A 
number of training consultants we interviewed stated that other priorities, such as 
registering new apprentices and marketing the program to new employers, had reduced 
their monitoring efforts. Only one district we visited had information on its monitoring of 
apprentices and employers. 


To make efficient use of limited resources, monitoring efforts need to be based on risk. For 
example, the program’s new information system, when completed, could be used to capture 
the information needed to identify specific industries or trades that are experiencing low 
program completion or satisfaction rates. This and other risk-based information would help 
training consultants focus their monitoring of apprentices and employers where it is most 


needed. 


For in-class training providers, the new information system is expected to facilitate periodic 
reviews of pass rates for each in-class training provider and trade. This and other outcome- 
based information will assist the Ministry in monitoring the performance of training 
providers. 
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Recommendation eee eee 


To better ensure the quality of apprenticeship training and compliance with 
training requirements, the Ministry should monitor the performance of : 
employers and in-class training providers. Such monitoring should include: — 








e on-site visits by field staff to employers and training providers with 
identified performance problems; and 

e tracking of the extent and results of monitoring visits to ensure any 
necessary corrective action is taken. 





Ministry Response 


We agree with this recommendation. Presently, field staff do some monitoring 
when they visit employers and training providers. The apprenticeship 
information system, now in development, will provide a case or portfolio 
management system for staff. Portfolio features include workload monitoring, 
bring-forward or follow-up reminders, and support material for assessment. 
The features are designed to enhance monitoring activities while assisting 
staff and management with workload management. To achieve maximum effect 
with these visits and in order to incorporate the information-gathering and 
tracking ability of the new information system, we will be developing a more 
comprehensive and risk-based monitoring policy and process that can be 
effectively implemented within existing resources. | 











Enforcement of Legislation on Restricted Trades 
Under the Tiades Qualification and Apprenticeship and the Apprenticeship and Certification 


acts, 20 trades in the construction, motive power, and service sectors have been designated 
as restricted. Only a certified tradesperson or a registered apprentice may work in these 
trades. Restricted trades include such occupations as automotive technician or construction 
electrician. To ensure consumer protection and workplace safety, it is important that 
individuals working in such trades are properly qualified and trained. 


Both acts allow the Ministry to inspect workplaces to ensure only qualified individuals are 
working in restricted trades. In 1993, an Order in Council delegated enforcement of 
certificate requirements for 19 of these trades to the Ministry of Labour (the Ministry of 
Training, Colleges and Universities retained enforcement authority for hairstyling). 


Enforcement activities were included as part of the Ministry of Labour’s normal workplace 
inspections. 


No information-sharing protocol exists between the two ministries although there were 
plans to implement one in the 2001/02 fiscal year. Our discussions with ministry field staff 
indicated that the extent of communication and co-ordination with local Ministry of 
Labour field offices was generally not sufficient to ensure effective enforcement. While some 
field staff commented that the two ministries had participated in sweeps of selected trades 
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and employers, other staff had little contact with local Ministry of Labour staff to determine 
the extent of enforcement activities. The Ministry had not clarified its expectations for 
enforcement activities to be conducted by its field staff. 


Also, there are other workplace inspection functions, such as those carried out by the 
Ministry of Transportation in the automotive sector and the Electrical Safety Authority in 
the construction sector. The focus of these inspections is more on consumer protection than 
workplace safety, which is the primary focus of inspections conducted by the Ministry of 
Labour. The Ministry has not established any protocols for sharing information about their 
activities. 

Consequently, the Ministry did not know the extent to which uncertified individuals might 
be working in restricted trades. Currently, only anecdotal information provided by 
stakeholder groups and complaints to field offices provide information on this problem. 
Effective enforcement not only protects consumers and workers but also maintains the value 
of entering a restricted trade, thereby increasing the likelihood that apprentices will 
complete their training. 


We noted that registrations in restricted trades have been increasing over the past four years, 
but there has not been a corresponding increase in the number of registered apprentices 
writing the required examinations to obtain their certificates of qualification. We also noted 
that, over the past three years, school enrolments for two restricted trades—automotive 
technician and auto body repairer—have declined by 12% and 26% respectively, despite 
increasing apprenticeship registrations in those trades. If there is little enforcement of 
apprenticeship requirements for restricted trades, apprentices may question the value of 
completing their programs. 


In implementing its new apprenticeship information system, the Ministry needs to capture 
information about the extent of non-compliance in specific trades and workplaces to help 
focus enforcement efforts where they are most needed. 








Recommendation 


To help reduce the extent of uncertified individuals working in restricted 

trades, the Ministry should: -_ _ 

¢ establish information-sharing protocols with the Ministry of Labour and 
other organizations that conduct safety inspections; 

* train field staff on ministry expectations for enforcement across the 
province; _ 

e use its new information system to help focus enforcement effort on 
industries, trades, and workplaces where the risk is greatest; and 


* monitor the impact of enforcement activities on apprenticeship program 
results. 
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Ministry Response | 

We agree with this recommendation and will be establishing an information- 
sharing protocol with the Ministry of Labour. There are also opportunities for 
ministry staff to benefit from an information-sharing protocol with the Ministry 
of Transportation, which enforces the restricted-certification skill-set programs 
for air-brake adjustment and truck-tire maintenance and may be in a position to 
identify mechanics operating without the required apprenticeship certification 
(although it has no statutory enforcement authority in this latter situation). 


The Ministry will begin negotiations with the Ministry of Transportation in the 
fall of 2002 to enter into an appropriate information-sharing agreement. The 
Ministry notes that the Electrical Safety Authority (ESA) currently does not 
have any legal authorization to enforce compulsory certification in the 
electrical trades. However, the Ministry will continue its work with the ESA and 
the Ministry of Consumer and Business Services, which has ministerial 
responsibility for the ESA, to develop an appropriate role for the ESA in 


enforcement activities in the electrical trades. 


The Ministry will review its current guidelines and practices, which will be 
amended as required by March 2003. All field staff will receive training 
regarding their responsibilities with respect to enforcement during 2003/04. 


The apprenticeship information system will provide the Ministry of Labour and 
the Ministry of Transportation with access to information as appropriate. This 
will enable recording of enforcement activities by these ministries and allow 
them to establish the currency of tradespersons certificates. Appropriate items 
will be directed to ministry staff for evaluation. 


The Ministry will review enforcement activities and monitor program results to 
determine if a linkage between enforcement activities and results can be made. 











Effectively Assessing Prior Learning and 
Academic Readiness 


As stated earlier, apprentices are generally required to complete an in-school portion of their 
training to attain certification. The minimum educational prerequisite for most 
apprenticeship training is Grade 12. To assist apprentices, the Ministry has implemented 
two optional learning assessment tools. One tool provides apprentices with recognition for 


relevant prior learning, and the second assists apprentices who may need help preparing for 
in-school training. 


PRIOR LEARNING 


Prior learning assessment is a process for identifying and recognizing what a person knows 
and can do. For example, an apprentice may have previously completed some or all of the 
required in-school components for a trade while attaining a college diploma. To recognize 
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prior learning and reduce unneeded in-school training, thereby accruing savings both to 
the Ministry and the apprentice, apprentices and non-apprentices who possess the entry 
requirements for a trade are eligible to write exemption tests. Where exemption tests are not 
offered, ministry staff can grant an exemption from in-school training if an apprentice can 
provide satisfactory documented proof of prior learning. 


At the time of our audit, exemption tests were offered for only 20 of the 92 trades that 
require in-school training. Ministry field offices administer standardized exemption tests for 
7 of the 20 trades. Colleges offer exemption tests for the remaining 13 trades, although 
many of their tests do not have standardized content or passing standards. 


The Ministry did not formally monitor the use of exemption tests offered either by colleges 
or by field offices to ensure they were being used effectively. Specifically, the actual use, 
number of attempts, and pass/fail rates needed to be monitored for each field office and 
college so that differences among field offices and colleges can be identified and investigated 
and corrective action taken where necessary. 


ACADEMIC READINESS 


To help reduce the risk of in-school failure in the apprenticeship program and thereby 
minimize the financial loss to the Ministry and the apprentice and time lost to the employer, 
the Ministry developed the Evaluating Academic Readiness for Apprenticeship Training 
tool (EARAT). The tool has three components that registered or prospective apprentices 
may find useful—a skills inventory, skill assessment tests, and a self-study guide. The tool can 
be used to identify areas of academic weakness that need to be addressed prior to 
attempting in-school training. It also provides information that allows prospective 
apprentices to make better decisions about whether a specific trade will meet their career 
expectations and whether they have the interest and aptitude to be successful. 


However, we found that the Ministry had not provided field offices with direction on the 
use, distribution, or administration of EARAT products. Further, the Ministry does not 
formally monitor field office use of EARAT products. Such monitoring would enable the 
Ministry to interpret trends in program completion rates and implement corrective action 
where necessary. 





-— 


Recommendation 


To better ensure that the Ministry’s learning assessment tools for the 
Apprenticeship program—both the assessment of prior learning and of 
academic readiness—are being used effectively, the Ministry should: 





¢ standardize and make available exemption tests for each applicable trade; 


° develop and communicate to ministry and college staff a standard policy on 
the use and administration of such tools; and 


e monitor the use and results of both tools and take corrective action where 


___ necessary. 


a a a a 
‘Training Division 303 

















| VFM Section 3.11 | 


| VEM Section 3.11 





i Se eee 


Ministry Response 


We agree with this recommendation. All existing trades with an in-school 
component will have standardized, ministry-approved exemption tests by 
2003/04. By the end of the 2002/03 fiscal year, exemption tests will be available 
in 87 of the 92 trades that currently require in-school training. Exemption tests 
will be developed for new apprenticeship trades as they are created. 


Policies for use of both exemption tests and EARAT (Evaluating Academic 
Readiness for Apprenticeship Training) tools are in place. In addition, EARAT 
skill inventories are available on the Ministry’s Intranet and all EARAT tools are 
available on compact disk. Ministry field staff have received training and 
ongoing support for the use of EARAT tools through Georgian College. The 
Ministry is currently reviewing the EARAT general policy to encourage more 
consistent and wider product use, including quarterly reporting from field 
offices on usage. 


Exemption tests administered by the Ministry are now being tracked. Tests 
shown to be flawed will be revised as needed. The use of EARAT tools is 
voluntary and client-directed and their quality is validated by stakeholders and 
revised as required. The Ministry is exploring the value of EARAT to individuals 
who are not already apprentices. 


es soa 











Managing Program Funding for In-school Training 


Total funding for in-school training was $58 million in 2001/02, 90% of which was 
provided to colleges. The remaining 10% was provided to private training deliverers, such 
as unions, based on actual attendance. Colleges receive funding for full-time students based 
on planned attendance and for part-time students based on actual attendance. A daily per 
diem rate ($59.81 in 2001/02) is used to determine each training deliverer’s funding 
entitlement. Funding for full-time students that is based on planned activity was introduced 
in 1992/93 to provide stable funding levels and to acknowledge that colleges incur certain 
costs whether enrolment is higher or lower than planned. 


We found that, since our last audit of this program in 1996, the proportion of planned seats 
actually filled by apprentices had increased from about 88% to 96%. This improvement is 
due to better forecasting of enrolment as well as increasing part-time enrolment. 


As part of its annual funding agreements, the Ministry requires colleges and private training 
deliverers to submit audited financial statements to provide assurance to the Ministry that 
per diem rates paid reflect the actual cost-per-day for training an apprentice. However, we 
interviewed staff from several colleges who indicated that the audited financial statements 
did not sufficiently reflect their full delivery costs. We also noted that the audited statements 
of many colleges were long overdue and that ministry staff were not reviewing the ones that 


had been received. Thus, the value of these financial statements for determining costs per 
day appears limited. 
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Also, as the Ministry begins to introduce more outcome-based performance measures for 
the apprenticeship program, it must find ways to link funding to the quality and outcomes 
of the training provided. Outcomes to consider include in-school completion rates, 
apprentice and employer satisfaction, and effective use of assessment tools. 





| Recommendation 


To help ensure that fanding levels for r in-school apprenticeship training are 
appropriate, the Ministry should: — 


¢ work with training providers. to develop fi financial reporting t that t reflects the 
actual cost of program delivery; and — 


e introduce funding that is linked to tte’ provision ~ training that recillist in 
positive obfoanes for Apprentines 2 and bombs _ 


Ministry Response 


We agree with this feconmendation. ‘on anodet 1, 2002, the Minion. : 
introduced a new funding model that involves the introduction of classroom — 
fees for apprentices attending the in-school portion of their training. As part of 
the on-going evaluation, the ety will undertake « a review ver Me ue sorool 
training-delivery funding model. -.. _ 


The Ministry i is developing a (Contouous lapesuoncas Porprienee | 7 
Management System for the Ontario apprenticeship system. The system is 
targeted for completion b y January | 1, 2004. Part of that system is key 
performance indicators. For example, information will be provided that ue 
allow the Ministry to evaluate pass/fail results by trade, by class, by level of 
schooling, and byi funded training-delivery agent. in order to establish a 
‘benchmark that would result i ina key performance indicator. The Ministry — 
expects that the measures will result i in i ponitive Caries pele tn made ae 








funded ‘training deliverers. 








LITERACY AND BASIC SKILLS PROGRAM 


The most recent /nternational Adult Literacy Survey, conducted in 1994, found that 
approximately 20% of the adult Ontario population did not have the basic literacy skills to 
meet workplace and daily living requirements. The Literacy and Basic Skills program 
supports and funds some 200 literacy agencies, including not-for-profit community groups, 
school boards, and colleges that provide services at some 300 sites across the province. The 
Ministry provides approximately $55 million annually for service delivery and 
approximately $5 million for service development. 


The services provided are intended to help adults improve their literacy skills so that they 


can enter or re-enter the workforce, go on to further education or training, and/or gain 


eee 
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independence. Accordingly, the program focuses on adults who are unemployed, with 
special emphasis on those receiving social assistance. To be eligible for services, a client must 
be at least 19 years old, out of school, and assessed as lacking the literacy skills necessary to 
find and keep employment or meet everyday needs. Learners must be able to demonstrate 
progress through their ability to complete exercises related to their goals. 


Tracking and Reporting Participant Outcomes 


Information about outcomes is essential for assessing and improving the performance of 
both literacy delivery agencies and the program overall. Performance indicators enable the 
Ministry to demonstrate whether the delivery agencies and the program overall are 
achieving the intended results and to link funding decisions to those results. 


In 1999/2000, the Ministry established a performance measure for tracking outcomes for 
participants—the percentage of participants who got a job or went on to further education 
or training as measured three months after the participant completes or leaves the program. 
In the Ministry’s Business Plan for 2000/01, the Ministry stated its commitment to have its 
delivery agencies provide 65,000 clients with initial services, such as information and 
referral. Of those 65,000 clients about 44,000 (68%) were to receive more intensive, 
longer-term instructional services. Of those 44,000, about 31,000 (70%) were to obtain 
employment or go on to further education or training. 


In its Business Plan for 2001/02, the Ministry increased the target number for clients in all 
three areas, but did not report publicly on the actual results for 2000/01. Internally, 
division management reported the actual 2000/01 program accomplishments as “62,800 
clients received service and 80% of participants got a job or went on to further education or 
training.” This could lead one to assume that 50,200 clients (80% of the 62,800) had a 


positive outcome. However, these reported results are misleading for several reasons. 


First, the actual 2000/01 program results did not reflect the fact that only 21,700 of nearly 
40,000 clients who received intensive services during the year actually completed or left the 
program. According to the agencies we visited, some clients require several years to complete 
their goals. However, the Ministry's outcome target did not take into account the number of 
clients who entered the program in prior years and the number that remained in the 
program at the end of the year. 


Second, in order to determine the rate of positive outcomes, all clients are to be surveyed 
three months after completing or leaving the program. However, agencies did not use 
consistent practices in determining which clients to survey. Of the agencies we visited, one 
surveyed participants who had completed their learning plans, another attempted to 
contact only participants who had been in the program for a minimum of six months, and 
only one was properly surveying all clients who had completed or left the program. As a 


result, delivery agencies on average attempted to survey only 60% of the 21,700 clients who 
completed or left the program in 2000/01. 
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Third, agencies were unable to contact about 28% of the clients they attempted to survey. 
Of the approximately 9,200 clients who were contacted, the agencies reported that about 
7,400 clients had a positive outcome, an 80% success rate. 


Excluding the approximately 3,800 lost contacts from reported results distorts reported 
results. For instance, one agency we visited had lost contact rates of 75%, 46%, and 63% 
over the past three fiscal years. Clearly, the outcomes reported by this agency are less reliable 
than those reported by agencies that had far fewer lost contacts. 


Also, at the time of our audit, the Ministry was unable to track the length of time clients 
stayed in the program. Length of program participation, combined with other available 
information such as client profiles, would help the Ministry identify trends in helping 


participants reach their goals as well as areas needing corrective action. 














Lees 
Recommendation 


To strengthen accountability and provide a sound basis for making informed 
funding decisions about its Literacy and Basic Skills program, the Ministry 
should: 


* ensure all program delivery agencies consistently conduct and report the 
results of their participant outcome surveys; 


* require that all program delivery agencies take steps to minimize lost 
contacts and to report them as part of program performance; 


e track and report the length of time clients remain in the program; and 


¢ report actual performance results in its Business Plan to permit a 
comparison with its commitments. 


Ministry Response 


We agree with this recommendation, and steps are being taken to ensure all 
program delivery agencies are consistently conducting follow-ups for all 
learners after leaving the program. The definition will be clarified in the Literacy 
and Basic Skills Program Guidelines, and ministry staff will reinforce 
compliance in program visits. 


Lost contacts will be included in the calculation of outcomes beginning with 
our reporting in 2003/04. Lost contacts will also be factored into program 
performance measures as they are being developed and introduced over the 
next two years. 


With the implementation of the information management system, the Ministry 
can now generate a report that tracks the length of time clients remain in the 
program. In 2002/03, the Ministry will establish baseline data from 2001/02 and 
will continue to track durations from that point on. 


The Ministry has begun the process of developing program outcomes. In the 
Ministry’s 2002/03 Business Plan, the performance targets have been stated 
L 
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more clearly to ensure transparency and clarity of understanding. Specifically, 
a target has been established for the number of clients accessing information 


and referral services separate from the number of clients accessing the more 
intensive training-related services. This will facilitate future reporting on 








program outcomes. | 


Linking Funding to Performance 


The Ministry negotiates annually with each service delivery agency to establish the number 
of contact hours to be provided. Contact hours represent the total amount of time that an 
agency is planning to spend delivering literacy services to all of its clients. The ministry- 
established funding ranges, within which each agency's approved funding allocation must 
fall, are based on the negotiated level of contact hours. Thus, funding 1s based on the 
amount of services to be provided. The Ministry needs to establish a specific link between 
funding levels and the quality and effectiveness of services in meeting client needs. 


Also, the level of funding is not sufficiently responsive to changes in activity levels, either in 
total or at individual agencies. Specifically, in the two fiscal years since this approach to 
funding was introduced, agencies have only provided approximately 85% of approved 
contact hours although they spent 97% of their funding. In 2000/01, for example, delivery 
agencies planned to provide 6.9 million contact hours for the $57.9 million in funding they 
were allocated, but they actually provided only 5.9 million hours for the $56.2 million they 
spent. Consequently, while the approved cost-per-contact hour was $8.35 based on total 
funding allocated, the actual cost-per-contact hour was $9.38 in that year, about 12% 
higher than planned. 


To determine whether funding provided to service delivery agencies was being adjusted in 
cases where actual service provided was significantly different than the expected level of 
service, we reviewed the funding and reported activity for a sample of sites for 2000/01. We 
noted that 37% of these sites only provided between 52% and 83% of the approved 
contact hours yet all of them had spent virtually all of the funds provided by the Ministry. 
Furthermore, for some of the sites we reviewed, this was the case in both years reviewed, as 


illustrated by the following table. 


Examples of Underperforming Agencies 














Source of data: Ministry of Training, Colleges and Universities 
Prepared by the Office of the Provincial Auditor 
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Recommendation 





To help ensure that funding to delivery agencies for the Literacy and Basic 
Skills program is appropriate and equitable based on the level and quality of 
services provided, the Ministry should implement a funding model that: 


¢ sets out the conditions and process which will result in adjustments in 
funding; and : 


* recognizes whether delivery agencies have been successful in helping their 
clients achieve positive outcomes. ; : 





Ministry Response - 


We agree with this recommendation. The Ministry has already begun a review — 
of its funding model. The revised funding model will be based on a principle of 
transparency and will be linked to performance standards to ensure that 
quality and effective services are provided. Benchmarks will be established 
and funding levels adjusted based on performance and circumstances. 
Introduction of the new funding model will occur late in the current fiscal year 
| with implementation in November 2003 for the 2004/05 business planning year. _ 














Monitoring Delivery Agency Performance 


The Ministry requires that its field consultants visit each agency approximately every 12 to 
14 months to evaluate its progress towards meeting ministry Business Plan commitments 
and its level of adherence to program guidelines and standards as well as the overall quality 
of program management. The agency visits are an important means of monitoring agency 
and program performance given that performance reporting on the quality and 
effectiveness of literacy services is under development. Consultants determine which 
agencies to visit based on: the time elapsed since the previous visit, as well as the consultant’s 
ongoing dialogue with the agency, awareness of specific issues, and overall impressions of the 
agency's success. 


We found that the Ministry's program for agency visits needed to be strengthened. 
Specifically, the program lacked: 


¢ a formal assessment of risk factors, such as complaints, financial- and performance- 
reporting problems, failure to meet contact-hour targets, and the results of previous 
visits; and 

* sufficient oversight of the process to ensure that problems identified were acted on in a 
timely manner. 


At the time of our audit, the Ministry was developing an assessment tool to help field 
consultants assess how well agencies were performing. 
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In addition to effective agency selection, the Ministry needs to maintain an accurate record 
of all agency visits completed and the status of any corrective actions required. At the time of 
our audit, the Ministry had begun to maintain a summary of agency-visit activity but it had 
not been completed. 





Recommendation 


To more efficiently and effectively ensure that field consultants and delivery 
agencies for the Literacy and Basic Skills program are meeting its 
expectations, the Ministry should: 


© ensure that field consultants formally assess the risk of performance 
problems when selecting and conducting monitoring visits to delivery 
agencies; and 

© track and summarize the results of all monitoring visits to determine 
whether visits are being conducted as expected and whether corrective 
actions are being taken when problems have been identified. 


Ministry Response 


We agree with this recommendation. In 2002/03, the Ministry will formalize the 
risk-assessment component used for determining the need and frequency of 
agency visits. The risk-assessment component will be reviewed and refined as 
performance standards are implemented. 


The Ministry will complete its tracking system to ensure all agency visits 
comply with our risk-assessment approach and will take corrective action as 
required. 











ACQUISITION AND MANAGEMENT OF 
CONSULTING AND OTHER SERVICES 


Since the 1998/99 fiscal year, the Ministry has provided funds to two third-party, not-for- 
profit agencies totalling approximately $11 million to undertake significant projects and to 
purchase related consulting, training, information technology, and other services to improve 
delivery of the Job Connect and the Literacy and Basic Skills programs. In each case, the 
agency received transfer payments from the Ministry to administer and pay for services on 
the Ministry’s behalf: In doing so, the Ministry did not follow the prudent purchasing and 
approval policies that would have applied if these significant services had been acquired 
directly by the Ministry. It also paid about $600,000 in GST on these services that it would 
not have had to pay had it engaged the service providers directly. 


While the services were engaged to provide standardized systems, reports, and monitoring 
tools, as well as training and support for program delivery agencies, the Ministry did not 





S10 2002 Annual Report of the Provincial Auditor of Ontario 


SSS SSS SSS SSS 


adequately control the acquisition and management of these services. We had concerns with 
the way the services were acquired, the way funds were advanced, and the way the contracts 
were managed. The Ministry could not demonstrate that value for money was obtained for 
the services provided. 


Information Systems Development Projects 


JOB CONNECT 
In the 1998/99 fiscal year, the Ministry asked a third-party, not-for-profit agency 


representing about half of the program delivery agencies to co-ordinate a project to develop 
and implement the Ministry's Job Connect information system. The not-for-profit agency 
then hired a company to co-ordinate the project without a competitive process. The 
company advised us that it hired a systems development firm through a selection process, 
but we were unable to review the process because the company was not required to retain 
the relevant documentation for more than three years. The systems developer reported 
directly to the company. From 1998/99 to 2001/02, the third-party, not-for-profit agency 
paid the company a total of $4.4 million in ministry transfer-payment funds, including the 
billings from the systems development firm ($1.1 million in 1998/99, $593,000 in 
1999/2000, $1.8 million in 2000/01, and $920,000 in 2001/02). In addition to the 
original development project, the Ministry’s annual funding for this project has included 
amounts for functional enhancements and maintenance of the system as well as delivery- 
agency training on the use of the system. The Ministry was planning to provide further 
funding in this regard in 2002/03. 


The Ministry requested yearly proposals from the company between 1999/2000 and 
2001/02. The proposals included planned project deliverables and corresponding costs. 
The annual agreements with the not-for-profit agency itemized the project deliverables the 
company was to provide but did not provide corresponding costs. The costs were approved 
as a lump-sum amount only. The annual audited financial statements requested by the 
Ministry provided assurance on the total project expenditures incurred by the company but 
not on costs incurred for each project deliverable. Thus, the Ministry could not ensure that 
those costs were reasonable and appropriate. 


LITERACY AND BASIC SKILLS 


In March 2000, the Ministry asked a not-for-profit service agency with information- 
technology and literacy-sector experience to lead development of the Ministry’s Literacy and 
Basic Skills information management system. In this case, the not-for-profit service agency 
also acted as the project manager for the contract. In August 2000, the agency selected a 
consulting firm to develop the system based on a bid price of $700,000. In November 
2000, the agency submitted a more comprehensive project proposal that included systems 
development, training, project management, technical support, and administrative 
overhead costs totalling $3.8 million. The system development component increased from 





‘Training Division Slit 





“VM Section 3.1 | 





| VEM Section 3.11 | 


ns 


$700,000-—the original bid—to approximately $1.6 million. The remainder of the 
proposal, $2.2 million, was for project management services, delivery agency training, and 
technical support to be rendered by the not-for-profit service agency that was selected 
without competition. The Ministry approved the $3.8 million project proposal in 
December 2000. 


As at March 31, 2002, the total costs billed were about $3.6 million, but the system was not 
yet complete. System development costs had increased to $2.3 million, a $1.6 million 
increase from the August 2000 proposal. Three factors increased development costs: 


¢ The project deliverables had not been finalized before the request for proposal (RFP) 
was issued in August 2000. The request for proposal instructed bidders that “the user 
requirements and preliminary data designs are still in progress. No decision has been 
made on what data identified will actually form part of the LBS [Literacy and Basic 
Skills] information management system.” Poorly defined project deliverables forced one 
bidder to respond “it is difficult to estimate the development work to be done and 
therefore our work effort.” 


© The Ministry did not direct the project manager to include the second phase in the 
REP in August 2000, but it was included in the November 2000 project proposal 
received from the project manager. The Ministry eliminated any possible economies of 
scale from bidders by not including both phases in the original RFP or by conducting 
another RFP. 


© The system development consultant was allowed to add 35% to its original bid to allow 
for unanticipated changes even before the project had begun. 


As with the system consulting service contracts we reviewed for Job Connect, the audited 
statements provided only the total costs incurred by the agency. Given the limited financial 
information provided, the Ministry could not compare what it paid for each deliverable to 
what it had approved for each deliverable. 


OBTAINING APPROVALS AND CONTROLLING COSTS 


Both of these projects and related consulting contracts were approved and awarded within 
the Ministry’s transfer-payment approval authority. Normally such significant information 
systems development projects require the approval of the Minister and Management Board 
of Cabinet. However, by providing transfer-payment funds to third-party agencies, these 
approvals were not obtained. That approval process is designed to ensure projects have been 
thoroughly justified and that proper competitive acquisition and project management 
processes have been established. Government directives also require specific senior 
management and central agency approvals to be obtained before awarding consulting 
contracts without a competitive process. However, the Ministry was unable to provide any 
documentation justifying or approving any exemption from following a competitive process. 
An open and transparent, well-defined, competitive process was not followed, and without a 


are process, the Ministry could not demonstrate that it obtained value for the funds 
expended. 
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In contrast, we noted that the Ministry did obtain all required approvals and followed a 
competitive process before awarding the consulting contract or releasing funds for the 
development of its new apprenticeship information system. Ministry operating funds were 
being used to develop the system. 


Computer Acquisition Projects 


LITERACY AND BASIC SKILLS PROGRAM 


The Ministry hired the same not-for-profit service agency managing the Literacy and Basic 
Skills program information systems project to oversee two projects to help program delivery 
agencies acquire new computers and software. The first computer acquisitions were to 
support program administration and the new information system, and subsequent 
acquisitions were to support service delivery to participants. The projects commenced in 
1999/2000 and 2000/01, respectively, and total funding allocated was $6.1 million: 
$900,000 for services, such as assessing the equipment and training needs of the agencies as 
well as for overall project management and administration; and $5.2 million for the 
acquisition of computers. We reviewed the two projects and noted the following concerns. 


The Ministry provided a lump-sum payment to the agency at the commencement of each 
project, rather than flowing funds only when required. At the completion of the two 
computer distributions, the agency still held approximately $1.1 million in ministry 
funding. The surplus funding arose for two reasons. First, the Ministry over-funded the 
agency by $500,000 even before the two projects began by miscounting the number of 
agencies to be funded. Second, the delivery agencies spent $600,000 less than expected to 
acquire the necessary equipment. 


In March 2002, the Ministry instructed the agency to use approximately $900,000 of the 
$1.1 million surplus for other projects, such as purchasing additional computers, 
developing training courses for the agencies, and constructing a technical support area at 
the agency’s facility. These additional projects were to be completed by March 31, 2002. 


No detailed accounting of actual expenditures was obtained by the Ministry. For example, 
we determined that the agency billed and was paid an additional $48,000 over and above 
the proposed and approved project administrative fees of $190,000. For the second 
computer distribution project, the approved fee was based on a percentage of project costs 
as well as a percentage of the costed computer equipment purchases. As a result, the 
approved administrative fee increased from $40,000 (15% of costs, excluding computer 
purchases) for the first project to $150,000 (51% of costs, excluding computer purchases) 
for the second project. The Ministry could not provide any support for approving the 


increase. 
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Job Connect Coaching and Other Consulting 


Services 


In 1999/2000, the Ministry requested another consulting firm to submit a proposal to 
provide professional coaching services to its Job Connect delivery agencies. Again, no 
competitive selection process was followed. Between 1999/2000 and 2001/02, the not-for- 
profit agency involved with the Job Connect information system project paid the consulting 
frm over $2.1 million from ministry transfer payments, $700,000 in each fiscal year. While 
most of the money was used to pay for coaching services provided to agencies, the 
consultant also assisted the Ministry directly by developing an agency evaluation framework 
and preparing new program guidelines. 


GST Payments 


The Ministry is GST-exempt when paying for services. However, the consultants were 
billing the not-for-profit agencies for services rendered, so GST was included in their 
billings. By using not-for-profit agencies, the Ministry has paid approximately $600,000 
more for these consulting services than it would have had to if it contracted directly with 
these consultants. 


For the Literacy and Basic Skills program, the not-for-profit agency correctly billed the 
Ministry 3.5% GST rather than the full 7% since the third-party administrator is a non- 
profit organization and is entitled to a 50% refund of GST paid. For services related to Job 
Connect, however, the not-for-profit agency included the full 7% billed by the consultants 
in its billings to the Ministry. We estimated that, since 1998/99, the Ministry paid the 
administrator $235,000 for GST that the administrator did not have to pay. 





[ 
Recommendation 


To better ensure that value for money is achieved in acquiring consulting and 
other services, the Ministry should: 


e comply with government policies for the acquisition of such services 
regardless of the source of funds used to acquire them; and 


e recover any funding provided to transfer-payment agencies that relates to 
GST that they were not required to pay. 


Ministry Response 


We agree with this recommendation. For the Job Connect system, the process 
undertaken in 2001/02 incorporated more prudent controls and processes for 
the acquisition of such services for 2002/03. The budget awarded to the 
successful bid is very comparable to systems development and support costs 
in the previous contracts and was identified by the Ministry’s Information 
Technology Branch as meeting a standard of value for systems development. 


[a 
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For the Literacy and Basic Skills (LBS) system, all subsequent systems 
development work will be fully scoped and tendered by the third-party service 
agency. 





The Job Connect information system supports the delivery of consistently 
high-quality services to individuals and employers across the province. It 
provides delivery sites with the agency-level program data they need to plan, 
evaluate, and adjust services to improve customer service in their 
communities. The LBS system is expected to provide similar benefits when 
completed. 


The data on individuals and employers, services provided, outcomes achieved, 
customer satisfaction, and costs supports the Ministry's ability to evaluate 
program policy and design in relation to its core business and strategic 
objectives. The systems also provide standardized, province-wide reporting 
mechanisms that allow the Ministry to manage the performance of funded 
agencies and to report program results effectively. 


A request for proposal (RFP) was not conducted for the delivery of coaching 
and other training services. The consulting firm that provided the coaching and 
training services was recommended by an advisory group representing the 
Job Connect delivery network based on their evaluation of an initial coaching 
and training project undertaken by the not-for-profit sectoral agency. 


The agency has been notified that an RFP must be conducted for network 
coaching or training development projects identified by the advisory group for 


The Ministry had already taken steps to recover GST that the not-for-profit 
agency was not required to pay. The Ministry will also ensure that GST is 
appropriately funded in future projects of similar scope and size. In addition, 
the Ministry will explore the implications of and options for developing 
policies/procedures related to the recovery of GST spent in the delivery of 
program services. . 
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CHAPTER FOUR 


Follow-up of 
Recommendations in the 
Special Report on 
Accountability and Value 
for Money (2000) 





It is our practice to make specific recommendations in our value-for-money (VFM) audit 
reports and ask ministries and agencies to provide a written commitment to take corrective 
action for publication in Chapter Three of our Annual Report. Two years after we publish 
the recommendations and related responses, we follow up on the status of actions taken by 
ministries and agencies with respect to our recommendations. 


Chapter Four provides some background on the value-for-money audits reported on in 
Chapter Three of our Special Report on Accountability and Value for Money (2000) and 
describes the current status of corrective action that has been taken to address our 
recommendations since that time. The sections in this chapter consist of follow-up reviews. 


Our follow-up reviews are planned and conducted to provide a moderate level of assurance 
on the extent and adequacy of corrective actions taken by ministries or agencies. The reviews 
consist primarily of inquiries and discussions with management; analyses of information they 
provide; and, where deemed appropriate, limited examination and testing of systems, 
procedures, and transactions. This is not an audit, and accordingly, we cannot provide a 
high level of assurance that the corrective actions described have been implemented 
effectively to fully resolve noted problems. Where these reviews identify new issues or 
concerns that need to be acted upon, these issues will be considered in our audit planning 
process. In any event, the results of the corrective actions taken or planned will be more fully 
examined and reported on through future audits. 


Overall, we concluded that a number of our 2000 recommendations appear to have been 


fully or substantially acted upon. However, there are many recommendations where action 
remains to be taken or is ongoing. 
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MINISTRY OF AGRICULTURE AND FOOD 


4.01—AgriCorp 


(Follow-up to VFM Section 3.01, Special Report on 
Accountability and Value for Money—2000) 
snes 


BACKGROUND 


Established by the AgriCorp Act, 1996, AgriCorp’s primary business is to administer 
insurance plans for the agriculture and food industry. At the end of the 2001/02 fiscal year, 
AgriCorp held assets totalling $413 million in its General Fund, Crop Insurance Fund, and 
Market Revenue Program fund. At the end of the 1999/2000 fiscal year, assets totalled 
$604 million. In 2001/02, AgriCorp’s General Fund paid administrative expenditures 
totalling over $12.2 million. In 1999/2000, it had paid $16 million. During 2001/02, the 
other two funds paid $322 million to compensate Ontario farmers for losses from reduced 
crop yields and low market prices. In 1999/2000, the two funds paid $180 million. 


In 2000, we concluded that AgriCorp did not have adequate procedures in place to ensure 
that its activities complied with legislation and corporate policies. In addition, AgriCorp 
failed to manage certain of its resources with due regard for economy and efficiency and, on 
a number of occasions, failed to safeguard the resources entrusted to it. Also, AgriCorp did 
not have the necessary governance and accountability procedures in place to ensure that the 
Corporation was well managed or to provide the information required to measure and 
report on its effectiveness. 


Some of our specific findings in 2000 were: 


* Inappropriately utilizing ministry funds, AgriCorp lost $325,000 in a speculative 
investment initiative to buy and sell bonds on a daily basis. AgriCorp also violated its 
fiduciary responsibility by attempting to transfer the loss to the Ontario Crop Insurance 
Fund. We had to intervene to reverse this transfer. 


* Contrary to legislation, on several occasions AgriCorp sought to remove money from the 
Ontario Crop Insurance Fund to pay for its administrative expenses. We had to 
intervene to ensure the Fund remained intact. 


* Without a proper business case and without tender, AgriCorp engaged an intermediary 
to place $14.5 million of reinsurance coverage with private insurance companies. 


* AgriCorp received little or no value for the $3 million it spent on information 
technology development projects that were poorly planned, controlled, and managed. 
In addition, information technology consultants were engaged without competition and 
provided with training at taxpayers’ expense, and several consultants were retained for 
periods ranging from five to 13 years at rates of up to $640 per day. 
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¢ AgriCorp engaged an investment advisor for a minimum annual fee of $400,000 
without competition. The advice received was of little value as the advisor repeatedly 
recommended investments to AgriCorp that were contrary to its legislation. 


Accordingly, we made a number of recommendations to improve the management and 
governance of AgriCorp and received commitments from AgriCorp and the Ministry that 
they would take corrective action. 


CURRENT STATUS OF RECOMMENDATIONS 


According to information received from AgriCorp, action has been taken on all of the 
recommendations made in our Special Report on Accountability and Value for Money 
(2000). The current status of each of our recommendations is outlined below. 


COMPLIANCE WITH LEGISLATION AND CORPORATE 
PROCEDURES 


Investments 


Recommendation 


To ensure that proper controls are in place to safeguard investments, AgriCorp should: 


* engage only in investment strategies that comply with corporate policies and the 
requirements of formal agreements; and 


© establish and ensure compliance with appropriate procedures to monitor investment 
activities. 


Current Status 


AgriCorp informed us that losses noted in our Special Report (2000) have been repaid to 
the Corporation’s General Fund. Also, the board of directors has approved new investment 
policies for both the General Fund and Crop Insurance Fund that comply with the 
AgriCorp Act, 1996. In addition, since October 2000, the Corporation has retained an 
external accounting firm that performs quarterly reviews of investment activities to 
determine compliance with the Corporation’s investment policies and reports directly to the 


board. 


Fund Administration 


Recommendation 


To ensure the integrity of all funds under its stewardship and to ensure that no fund benefits at 
the expense of another, AgriCorp should: 


° ensure that the Corporation’s activities are legislatively sound; 
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* implement policies for interfund transactions, including the payment of fair rates of 
interest on any interfund loans; and 


°* review administrative funding arrangements to properly deal with funding pressures. 


Current Status 


We were informed that, during 2000, AgriCorp transferred $2.1 million, plus interest, 
from its General Fund to the Crop Insurance Fund. This transfer represents a repayment to 
the Crop Insurance Fund for monies that had been used to pay for administrative expenses. 
Such expenses, it was determined, should have been paid from the General Fund. Also, in 
April 2001, the board of directors approved a policy with respect to interfund loans 
whereby a loan agreement would be drawn up for such loans. This would include 
identification of the lending and borrowing funds, the amount, interest rate, and repayment 


schedule. 


With respect to administrative funding arrangements to deal with funding pressures, in 
accordance with a new Memorandum of Understanding, the Corporation will negotiate 
with the Ministry of Agriculture and Food for any additional funding required on a case-by- 
case basis. 


Insurance Claims 


Recommendation 


Lo ensure that the proper indemnity payments are made in compliance with legislation and 
corporate policy, AgriCorp should: 


° manage the Optional Unit Coverage Program to ensure that there is no additional risk 
exposure to the Ontario Crop Insurance Fund: 


* equitably enforce the deadline dates for insurance applications and premium payments 
on all crops; 


° direct insurance adjusters to measure the acreage related to any claim for crop loss or 
damage; and 


* reassess procedures as necessary so that indemnities are paid only when actual yields are 
less than guaranteed yields. 


Current Status 


The Optional Unit Coverage program will continue to be offered on a pilot basis with the 
current program covering the 2002 crop year. AgriCorp informed us that an additional 
deductible—based on the number of units insured—has been implemented to reflect the 
increased level of risk for this program. 


In addition, AgriCorp informed us that it will continue to allow premium payments to be 
remitted after the harvest under third-party agreements because the premiums are 
guaranteed by the third party and producers cannot opt out of the program once they have 
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enrolled. Indemnity payments are made only after AgriCorp has received the insurance 
premium. 


To help insurance adjusters track crop yields and damage claims, AgriCorp informed us 
that it started using a geographic information system to evaluate 2001 crop losses. The 
Corporation will use this system to compare crop yields in a geographic area and investigate 
those that are not in line with the effects of weather events and adjacent farms in the area. 


AgriCorp implemented a pilot project in 2000 to test an alternative method of insuring 
forage crops based on a model that compares in-year rainfall to historic levels of rainfall to 
corroborate yield estimation. The pilot is being evaluated to determine the accuracy of the 
insurance coverage provided, the cost of administration, and customer acceptance. 


DUE REGARD FOR ECONOMY AND EFFICIENCY 


Information Technology Management 


Recommendation 


To ensure that information technology project management results in the effective execution of 


plans and achieves results economically, AgriCorp should: 


¢ develop a long-term strategic plan for information technology that reflects the 
operational needs of the Corporation; 


prepare a proper business case for all potential systems development projects, including 
an assessment of corporate needs, options available and a detailed cost/benefit analysis 


of each option; 


before proceeding with any systems development project, develop and receive board 


approval for a project plan outlining project deliverables and the nature and timing of 
milestones to be achieved; 


establish a rigorous monitoring program to ensure the achievement of milestones and 
the satisfactory completion of all systems development projects; and 


thoroughly evaluate any new sofiware applications before purchasing them to ensure 
that they meet corporate needs and can be implemented. 


Current Status 


In March 2001, the board of directors approved a three-year strategic plan that included, 
as a core element, strategic objectives for information technology. All major information 
technology purchases now require a business case that includes an assessment of corporate 
needs and available options and a cost/benefit analysis. In addition, the board of directors 
must now approve all purchases in excess of $100,000. 


As well, the Corporation has adopted systems development guidelines to help manage and 
monitor its information technology projects. AgriCorp informed us that project control, 
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reporting, and evaluation criteria will be part of the new methodology to help ensure that 
milestones are achieved and that systems development projects are completed. 


Consulting Services 


Recommendation 
To obtain value for funds epended on consultants, AgriCorp should ensure that: 


° consultants are engaged and contracts are renewed through a competitive process and 
any exceptions are adequately justified, documented, and approved; 


° consultants are not used to perform work other than that specified in their contracts; 


© consultants have the necessary skills to carry out their work assignments and not be 
trained at taxpayers” expense; and 


° assignments are formally evaluated upon completion. 


Current Status 


We were informed that the Corporation now follows a competitive process whenever 
consulting services are required. AgriCorp has also reduced its reliance on information 
technology consultants and is developing expertise internally. The Corporation has retained 
one long-term consultant because of the need for specific programming knowledge that is 
not readily available in the industry. However, the Corporation indicated that this contract 
would not be renewed after June 2002. In addition, in future, consultants will be 
responsible for their own training costs and all consultants will be formally evaluated at the 
completion of each assignment to assess their performance and qualification for future 
assignments. 


Investment Services 


Recommendation 


To ensure that necessary investment services are acquired economically, AgriCorp should: 

° assess the need for an investment advisor; 

* acquire investment services through an open and competitive process; 

° obtain legal advice regarding the terms and conditions of all major contracts; and 


® consider obtaining investment advice from within the government. 


Current Status 


AgnCorp informed us that they did not renew the contract with their investment advisor 
when it expired on December 31, 2000, and the Corporation now makes investment 
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decisions in-house. In addition, AgriCorp is seeking investment advice, when needed, from 


the Ontario Financing Authority. 


Reinsurance Coverage 


Recommendation 
To ensure that both its current reinsurance program and any future reinsurance arrangements 


are economical and appropriate, AgriCorp should: 


© determine whether the reinsurance program has resulted in any direct rate reduction to 
premiums or increase to investment yields; 


© consider obtaining expert advice to review the current arrangements and potential 
options; 


° prepare a business case for reinsurance that quantifies expected costs and benefits; 
° acquire any future reinsurance through a fair, transparent, competitive process; and 


© obtain board approval prior to entering into any future reinsurance arrangements. 


Current Status 


AgriCorp now prepares a business case for reinsurance coverage each year that includes an 
analysis of the costs and benefits, the effect on premiums, and alternatives. The board of 
directors is required to review and approve the business case before a competitive process Is 
undertaken for reinsurance coverage. AgriCorp also informed us that it has retained an 
actuarial firm to design and certify a rate-setting methodology that will ensure the 
sustainability of the Crop Insurance Fund and stabilize premium rates over time. 


Travel Expenditures 


Recommendation 


To ensure that employees are properly reimbursed only for work-related expenditures, AgriCorp 
should: 


not reimburse employees for expenses that are of questionable business legitimacy; 


develop clear guidelines outlining when the extension of hospitality at the Corporation's 
expense 1s appropriate; 


reimburse only those claims for reimbursement that are accompanied by proper 
documentation; and 


ensure that managers approve only travel expense claims that comply with corporate 
procedures. 
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Current Status 


We were informed that new procedures have been put in place for approving and 
processing travel claims. A new policy for extending hospitality has also been adopted to 
define what expenditures are permitted and reasonable in a business context. 


Purchasing 


Recommendation 


Lo ensure the economic purchase of goods, AgriCorp should purchase all goods through the 
purchasing department unless the use of petty cash or employee travel expense claims is justified 


Current Status 


AgriCorp informed us that it reinforced its purchasing policies and procedures with staff 
and clarified the appropriate use of petty cash and employee travel expense claims for the 
purchase of goods and services. 


Management of Movable Assets 


Recommendation 
10 properly control and safeguard its movable assets from loss, AgriCorp should: 


°* continually update its asset inventory list to ensure that the list reflects all purchases, 
disposals and reassignments of movable assets; 


° perform a periodic asset verification to identify discrepancies for subsequent follow-up 
and resolution; and 


* implement a termination checklist to ensure assets are retrieved from departing 
employees and consultants. 


Current Status 


AgriCorp indicated that it now maintains an up-to-date asset listing and will implement a 
process to perform periodic verifications of computer hardware, furniture, fixtures, and 
other assets. We were also informed that for the 2002/03 fiscal year, a revised termination 
checklist will be prepared and new procedures will be developed to ensure that all assets are 
returned when an employee leaves the Corporation. 


MEASURING AND REPORTING ON EFFECTIVENESS 


Program Effectiveness 


Recommendation 
Lo ensure that it is effective in meeting its legislated objectives, AgriCorp should: 
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© establish a clear vision statement that reflects tts legislated objectives; 


¢ develop performance measures that are linked to its established vision and objectives; 


and 


perform the assessments necessary to determine whether its operations are achieving its 
established vision and related objectives. 


Current Status 


A review of AgriCorp’s mandate was undertaken in October 2000 and a three-year 
business plan was approved in March 2001. The business plan for the fiscal years 2001/02 
to 2003/04 outlines AgriCorp’s mandate as specified in legislation, and the corporate 
mission statement clearly spells out the responsibilities and priorities of the Corporation. 


Through the business planning process for the 2002/03 fiscal year, AgriCorp plans to 
develop one-year performance measures that are specific, measurable, and timely. AgriCorp 
intends to seek input and agreement from the Ministry in developing and finalizing these 
measures. In addition, AgriCorp’s CEO and the Deputy Minister now meet quarterly to 
assess the Corporation’s progress toward meeting its business plan objectives. Monthly 
meetings between corporate and ministry staff have been initiated to discuss ongoing policy 
and operational issues and to review AgriCorp’s performance. 


Corporate Governance 


Recommendation 


To improve corporate governance, AgriCorp should: 


© review the board [of directors] and its committee structure and establish clear lines of 
communication and accountability; 


ensure that management provides timely, sufficient, and appropriate information for 
decision-making; and 


periodically assess the effectiveness of the board with respect to governance and the 
attainment of corporate objectives. 


Current Status 


AgriCorp informed us that, in order to improve corporate governance and establish clear 
lines of communication and accountability, the board of directors was restructured in 

June 2000. This restructuring resulted in a single, 12-member board structure, as opposed 
to the previous dual board structure. The Corporation also established two standing board 


committees with clearly defined roles and responsibilities—the Executive Committee and 
the Crop Insurance Committee. 


In November 2001, a new Memorandum of Understanding between AgriCorp and the 
Ministry was put in place detailing the responsibilities of the chief executive officer to the 
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board of directors, its chair, and the Deputy Minister. In addition, the Ministry or the 
Management Board of Cabinet may initiate periodic reviews of the Corporation, which 
would include reviews of the roles of the chair and Deputy Minister, the mandate of the 
Corporation, and its organizational structure. 


MINISTRY RESPONSIBILITY 


Recommendation 


To help ensure that AgriCorp is effectively fulfilling its mandate and that its operations are 
economic, efficient and in accordance with legislation, the Ministry should: 


°* update its Memorandum of Understanding with the Corporation to include any new 
monitoring procedures and Management Board of Cabinet requirements; 


¢ outline specific roles and responsibilities for the ministry representative on the board of 
directors; 


¢ develop and implement procedures to more effectively monitor the activities of the 
corporation; and 


* implement a periodic internal audit process to provide assurance that the Corporation 
fulfills its financial arrangements stipulated in the Memorandum of Understanding. 


Current Status 


In November 2001, AgriCorp signed a new Memorandum of Understanding with the 
Ministry that outlines the responsibilities of the Ministry and the Corporation as well as the 
ministry representative on the board of directors. One of the responsibilities of the Ministry 
representative is to perform an overall monitoring function of the Corporation and to 
provide the Deputy Minister with quarterly reports containing an overview of the Board’s 
decisions and summaries of financial and administrative matters to help the Minister 
determine whether AgriCorp is fulfilling its mandate. In addition, the Memorandum 
requires that the Ministry's Internal Audit Branch periodically carry out audits of the 
activities of the Corporation. 
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MINISTRY OF COMMUNITY, FAMILY AND CHILDREN’S 
SERVICES 


4.02—Child Welfare Services Program 


(Follow-up to VFM Section 3.02, Special Report on 
Accountability and Value for Money—2000) 


BACKGROUND 


The Ministry (at the time of our audit named the Ministry of Community and Social 
Services) is responsible for setting expectations for child welfare services through legislation 
and regulations, as well as for funding and monitoring services provided by Children’s Aid 
Societies (Societies) and taking corrective action where necessary. Services provided by the 
Societies include: investigating and assessing child abuse and neglect allegations, as well as 
providing counselling, guardianship, and many other types of services that protect children 
from neglect and physical, sexual, and emotional abuse. 


Program expenditures for the year ended March 31, 2002 were $879 million, 

$864 million of which was for transfer payments to the Societies. Program expenditures 
during 1999/2000 totalled approximately $654 million, of which $642 million was for 
transfer payments to the Societies. The Ministry estimated that 154,000 children from 


86,000 families received some type of service from a Society during the fiscal year ending 
March 31, 2000. 


Our principal conclusion in 2000 was that the Ministry did not have sufficient assurance 
that children in need were adequately protected because: 


© Societies could not always demonstrate that they conducted their assessments of children 
reported to be in need; 


¢ notall plans of service, which outline the actions that need to be taken to protect a child, 
were prepared or implemented on a timely basis by the Societies; and 


program outcome measures had not been developed and implemented. 


We also concluded that the Ministry’s new funding framework, which was to be fully 
implemented in the 2000/01 fiscal year, is a substantial improvement over the Ministry's 
previous method of determining the Societies’ program funding. However, the Ministry will 
not realize the full benefits of the new funding framework until direct and indirect service 


costs to be funded are linked to the nature and assessed cost of the underlying services to be 
received, 


We made recommendations to overcome these deficiencies, and the Ministry responded to 
our recommendations with commitments to take corrective action. 
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CURRENT STATUS OF RECOMMENDATIONS 


The Ministry is in the process of acting on many of our recommendations as detailed below. 


CASE MANAGEMENT 


Eligibility for Protection Services and the Protection 
Investigation 


Recommendation 


To help ensure that children reported to be at risk receive the appropriate protection in 
compliance with the Risk Assessment Model Jor Child Protection in Ontario, the Ministry 
should: 


* conduct periodic file reviews to assess compliance with case management standards in 
general and the Risk Assessment Model in particular; and 


° identify instances of non-compliance and ensure that the necessary corrective actions 
are taken on a timely basis. 


Current Status 


The Ministry developed a case-monitoring tool in December 2000 and conducted pilot 
reviews of a sample of child protection files from 13 Children’s Aid Societies between April 
and June of 2001. The resulting report, dated October 2001, identified areas where 
further improvement was required to achieve compliance with the Standards for Child 
Protection Cases, including the risk assessment model, and also identified the need for 
changes to the Standards for Child Protection Cases. 


Subsequent to these pilot reviews, the Ministry planned to conduct child protection reviews 
in all Children’s Aid Societies in 2002. Reviews commenced at two Societies in the first three 
months of 2002, but a labour strike in the Ontario government put further reviews on 
hold. The Ministry plans to resume these reviews at the remaining Societies commencing in 
October 2002. 


Protective Services and Plans of Service 


Recommendation 
To help ensure that minimum service expectations for all child protection cases are met, the 
Ministry should monitor the Societies to make sure that: 


° services for children determined to be in need of protection commence within the 


required 30 days of the initial referral; and 


° plans of service for these children are prepared and approved by a supervisor within 60 
days, as required. 
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Current Status 

The previously mentioned ministry pilot reviews identified a continuing need for better 
compliance to ensure that services for children determined to be in need of protection 
commence within the required 30 days of the initial referral and that the plans of service for 
these children are prepared and approved by a supervisor within 60 days, as required. 


As already noted, the Ministry is planning to conduct child protection reviews in all 
Children’s Aid Societies commencing in October 2002 to determine compliance with all 
requirements of the Standards for Child Protection Cases. 


Children in Residential Care—Crown Ward Reviews 


Recommendation 

To help ensure that services provided to Crown wards are appropriate and in compliance with 

regulatory requirements, the Ministry should: 

© issue directives in all instances where program regulations have not been complied with; 
and 


© monitor Children’s Aid Societies to ensure they act on all directives and confirm 
compliance with the directive to the Ministry within 60 days, as required. 


Current Status 


We were advised that the Ministry re-emphasized the need to issue directives for all instances 
where there is non-compliance with program regulations. In addition, the Ministry has 
developed and implemented an automated bring-forward system that provides prompt 
follow-up to ensure that Societies now act on directives issued within the required 60 days. 


A review by Internal Audit of a sample of Crown ward files found that the follow-up reports 
to the Ministry were generally issued within the required 60 days. However, the follow-up 
reports often did not provide specific action to address the concerns identified. There was 


also often no evidence of ministry review of the appropriateness of the responses provided 
by the Societies. 


Children in Residential Care—Non-Crown Ward Reviews 


Recommendation 


To help ensure that protective services provided to non-Crown wards are appropriate and in 
compliance with program requirements, the Ministry should: 


¢ implement a regular review process for non-Crown ward files; and 


° ensure instances of non-compliance with program requirements are communicated to 
the Children’s Aid Society and, where required, acted upon on a timely basis. 
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Current Status 

The Ministry has substantially addressed this recommendation. The Ministry's monitoring 
process for non-Crown ward cases was implemented in 2001, in which the Ministry now 
conducts annual reviews on a sample of non-Crown wards. The Ministry has issued 
guidelines for these reviews, as well as guidelines for issuing directives when the reviews 
identify non-compliance with legislative and regulatory requirements. Societies are required 
to report corrective action to the Ministry within 60 days. 


A review by Internal Audit of a sample of non-Crown ward files identified that directives 
were issued for areas of non-compliance. 


Extended Care and Maintenance Agreements 


Recommendation 
To help ensure the successful transition of Crown wards to independent living, the Ministry 


should ensure that: 


* Children’s Aid Societies regularly monitor youths who have extended care and mainte- 
nance agreements, as required by their agreements, and assess compliance with their 
personal goals; and 


* where the requirements of the agreements are not met, corrective action is taken by the 
Societies on a timely basis. 


Current Status 


The Ministry indicated that it considers the current accountability practices by regional 
offices to be sufficient to monitor the Societies’ practices with respect to the extended care 
and maintenance agreements of former Crown wards. As a result, the Ministry has not 
made any progress on this recommendation. 


Licensing of Children’s Residences 


Recommendation 


To encourage residential care operators to provide minimum acceptable standards of care to 
children, the Ministry should conduct and document licensing inspections and renew licenses on 
a more timely basis. 


Current Status 


The Ministry has implemented a licensing module into its Service Management 
Information System and intends to monitor the licensing review and renewal data through 
the system annually. 
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Serious Occurrences 


Recommendation 
To help safeguard children in care and gain assurance that all serious occurrences are 
appropriately dealt with, the Ministry should take the necessary steps to ensure that: 


© an initial notification report is prepared for every serious occurrence reported; and 


e written follow-up reports from the Societies are evaluated to determine whether appro- 
priate actions have been taken. 


Current Status 

The Ministry has made substantial progress in implementing this recommendation. The 
Ministry sent out revised standards and guidelines for reporting serious occurrences in July 
2000. These guidelines require that Societies inform regional offices of a serious occurrence 
within 24 hours, and the occurrence must be followed up with a written report within 
seven working days. Annual summaries and reports of analyses are also now prepared by the 
Societies and submitted to the regional offices. 


FINANCIAL ACCOUNTABILITY 


Funding Framework 


Recommendation 
To ensure that funding provided to Societies under the new funding framework is appropriate 
for them to deliver the required services, the Ministry should ensure that: 


residential care per diems are based on an assessment of what reasonable costs should be 


for each type and category of care; 


direct service cost funding is based on a ministry review and assessment of the underly- 
ing work performed; 
funding for indirect service costs is reasonable and appropriate for each Society; and 


revenue benchmarks are reasonable and attainable. 


Current Status 


The Ministry conducted an interim review of the benchmarks and design elements of the 
Child Welfare Funding Framework in 2000. Based on the results of this review, the 
Ministry revised the benchmarks used in the funding framework for 2000/01 for group 


care costs and revenue earned. 


In the Fall of 2001, the Ministry completed a more comprehensive review resulting in a 
number of draft recommendations for changes to the funding framework designed to 
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ensure flexibility in providing the most appropriate services, contain costs, and reduce 
inequities across agencies. 


Service and Financial Data Review 


Recommendation 
[n order to maintain the integrity of the Ministry's new funding framework, which is primarily 
based on caseload volume data, the Ministry should: 


° establish ongoing requirements for and conduct regular Children’s Aid Society caseload 
data reviews; and 


* adopt appropriate policies and procedures for such reviews. 


Current Status 


Regional offices are now instructed to conduct caseload data reviews for all Societies at least 
once a year, regardless of volume, by May/June of each year. 


The Ministry has developed and implemented the use of detailed checklists and 
questionnaires for these reviews. The reviews include an examination of a sample of case files 
in the non-residential and residential categories, as well as interviews with key management 
and financial staff; Concerns identified are to be monitored for corrective action where 
required. 


Quarterly Reporting 


Recommendation 


Ln order to ensure the timely identification, analysis, and follow-up of. significant in-year 
variances in expenditure and caseload data, the M; inistry should: 


* obtain the required quarterly report variance analysis on a timely basis and in suffi- 
cient detail, including the necessary explanations and corrective actions; and 


* clearly document its process for reviewing variances and, if necessary, approving the 
appropriate corrective actions. 


Current Status 


In 2000/01, Societies provided the Ministry with the first annual Service Management 
Plan, which included expenditure forecasts. The Societies are now instructed to submit 
quarterly reporting packages that capture detailed service and financial data that are to be 
reviewed by the Ministry and compared to the Service Management Plan. 
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Annual Program Expenditure Reconciliation 


Recommendation 

To ensure that it identifies and recovers ineligible expenditures and surplus funding, if any, the 
Ministry should: 

© receive, review, and approve all Annual Program Expenditure Reconciliation on a 


timely basis; and 


© ensure that Annual Program Expenditure Reconciliations and accompanying audited 
financial statements contain sufficiently detailed and comparable information to 
identify ineligible expenditures or surplus funding. 


Current Status 


The Ministry advised us that it has developed and implemented a revised fourth-quarter 
year-to-date and year-end Reconciliation Report that incorporates the Annual Program 
Expenditure Reconciliation. 


OTHER MATTER 


Information Systems 


Recommendation 


In order to facilitate the collection and analysis of performance data necessary to effectively 
administer the Child Welfare Services program, the Ministry should expedite the development of 


its proposed management information system. 


Current Status 


The Ministry has implemented the Fast Track Information System in all Societies for 
tracking high-risk families. 


The Ministry also plans to collect outcome and performance data on 10 key Child Welfare 
indicators and is to build these into a Comprehensive Child Welfare Information System to 
be developed at a later stage. The Ministry is currently working on a pilot project to collect 
data on three of these ten indicators. The Ministry plans to implement regular reporting 
requirements to collect all outcome data beginning in 2002/03. 
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MINISTRY OF CONSUMER AND BUSINESS SERVICES 


4.03—Project to Automate the Land 


Registration System (POLARIS) 


(Follow-up to VFM Section 3.03, Special Report on 
Accountability and Value for Money—2000) 





BACKGROUND 


In 1980, the Ministry (at the time of our audit named the Ministry of Consumer and 
Commercial Relations, but since renamed the Ministry of Consumer and Business Services) 
established POLARIS (Province of Ontario Land Registration System) to automate the 
province's land registration system. The project involves the conversion and automation of 
paper-based records to permit the searching and registration through computers of real 
property documents. In 1991, the Ministry transferred ownership and responsibility for the 
implementation and operation of POLARIS to Teranet Land Information Services Inc., 
since renamed Teranet Inc. (Teranet), a corporation owned jointly by the province, with 
40% of the voting shares, and a private-sector company, with 60% of the voting shares. As 
of March 31, 2000, approximately 2.5 million of the estimated 4.3 million properties in the 
province had been fully converted to electronic format. 


‘Teranet receives fees collected by the Ministry out of the Consolidated Revenue Fund for 
certain land-registration-related services that the Ministry processes using POLARIS. In 
addition, ‘Teranet receives fees directly for land-registration-related services provided 
remotely. In return, the Ministry receives royalties from Teranet. From Teranet’s inception in 
1991 to March 31, 2000, Teranet received $235 million from land-registrated-related 
service fees, of which Teranet had paid back $45 million in royalties to the Ministry. 


During our review, we identified a number of significant concerns with the Ministry’s 
arrangement with Teranet and the status of its project to automate the province’s land 
registration system, including: 


* In 1991, the POLARIS project had an anticipated 1999 completion date, but at the 
time of our review, ministry information received from Teranet indicated a 2010 project 
completion date. 


* Cost estimates to complete the project had substantially increased from the original 
1991 estimate of $275 million. As of April 1999, Teranet estimated that total costs for 
the project would be more than $700 million—an estimate based on Teranet’s 
assumption that certain cost-saving methods could be implemented. A consultant 
engaged by the Ministry to provide advice on the Ministry’s financial risk with respect to 
the project noted that under other less favourable scenarios, the total project costs could 
exceed $1 billion. 
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© The consultant also noted that the cost to convert the remaining 1.8 million properties 
using existing workflows and processes exceeds the anticipated revenues from 


conversions. 


© There isa risk that if the Ministry were required to terminate its agreement with Teranet 
and assume operation and control of the POLARIS project, it would have to address 
Teranet’s $280-million obligation to its bondholders. As well, the Ministry may have to 
compensate Teranet for all or part of the $300 million in costs Teranet has incurred. 


At the time, the Ministry advised us that it had not agreed to changes to the contractual 
obligations of Teranet for completing the project and that it was in the process of deciding 
on an appropriate course of action. 


In 2002, POLARIS was renamed the Electronic Land Registration System (ELRS). 


CURRENT STATUS 


Based on information we received from the Ministry during our follow-up in April 2002, 
the Ministry had renegotiated its arrangements with Teranet to address concerns regarding 
the project’s costs and completion date. The project is now estimated to cost $680 million, 
its completion date is expected to be September 2007, and its scope has been reduced such 
that only 87% of all properties are to be fully converted and automated. 


Comparison of Original POLARIS and Current ELRS Project Plans 


Under Under 


Original Renegotiated 
Plan (1991) Plan (2002) 














Implementation costs 







$275 million $680 million 


2007 


Source of data: Ministry of Consumer and Business Services 








Completion date 








Percentage of properties to be converted and automated 


After an extensive negotiation process, in March 2002 the Ministry and Teranet entered 


into new agreements, which entail major changes to the Ministry's old arrangements with 
Teranet and include the following: 


¢  Teranet is required to substantially complete the conversion and automation process by 


September 2007. The project’s completion has been redefined from converting and 
automating 100% of properties in the province to converting and automating about _ 
87% of properties; the agreements require that the 87% of properties include all 
properties in communities with a population of 5,000 or mote. The revised estimated 
number of properties in the province as of July 1, 2001 was 4.7 million, and 4.1 million 
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of these properties were expected to be fully converted under the reduced project scope 
by the new completion date. As of March 31, 2002, approximately 3 million properties 
had been fully converted and automated, leaving 1.1 million properties remaining to be 
converted and automated. 


* The new implementation plan, which forms part of the revised agreements, summarizes 
previous actual costs and expected future costs for the completion of the project as 
follows: 


Incurred POLARIS and Expected ELRS 
Project Implementation Costs 





Costs incurred from 1991 to March 31, 2002: 
Direct implementation costs 
ndd 


Yj 


Direct implementation costs 


Interest and deprec 


ye 





Source of data: Ministry of Consumer and Business Services 


* The completion date and costs and revenues are at risk if the assumptions on which the 
implementation plan is based do not prove to be correct. Under the new arrangements, 
a joint committee is to be established to monitor the progress of actual implementation 
against the implementation plan. Teranet is required to report quarterly to the 
committee on its costs, efforts, and progress, as well as how real estate market activity, 
which can impact the conversion timetable, compare to assumptions in the 
implementation plan. Should a substantial variance occur over several quarters, the 
committee may investigate the cause of the variance and make recommendations for 
corrective action. However, Cabinet authority is required to adjust the total cost of 
implementation or extend the completion date. 


* —Teranet, which holds a conditional, exclusive licence for providing automated land 
registration services, receives fees collected by the Ministry for basic services that are 
processed using ELRS such as registrations and certain ancillary transactions. In 2017, 
the exclusive licence is expected to end. The Ministry estimates that Teranet will have 
received almost $2 billion in fees for basic ELRS services up to 2017. In addition, it is 
estimated that Teranet’s other land-registration-related revenues over this period, such as 
fees charged for remote access to ELRS and other premium services, will total about 
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$1 billion. In consideration of Teranet’s estimated revenues up to 2017, the Ministry 
forecasts it will receive $466 million in royalties from Teranet over the same period. 
Teranet is not expected to pay any dividends to its shareholders during this period. 


The following table summarizes Teranet’s actual and expected revenues and 
expenditures for its core business related to land registration services from its inception 
to 2017, when its conditional exclusive licence is expected to end. 


Teranet’s Actual and Expected Core Business Revenues 
and Expenditures for Land Registration Services, 1991-2017 





















($ million) | 


Revenues 






Fees paid either from the Consolidated Revenue Fund or 

directly to Teranet for basic registration services using 

POLARIS/ELRS (land title registrations, searches, certain 

ancillary fees) 1,976 


Fees paid directly to Teranet for premium services (remote 
access, maps) 498 


Fees paid either from the Consolidated Revenue Fund or 
directly to Teranet for execution of writs searches 521 


Total Revenues 


Expenditures 





POLARIS/ELRS project implementation costs 680 
Investment in system development 160 
Investment in capital 124 
Operations 319 
Marketing 201 
Royalties to the Ministry of Consumer and Business Services 466 


Other 
Interest and taxes 





Total Expenditures 












Source of data: Ministry of Consumer and Business Services 


¢ The new agreements further clarify the Ministry’s rights should Teranet become 
insolvent or default on its obligations. If Teranet should, for example, fail to provide a 
minimum production capacity or willfully fail to perform according to the 
implementation plan (in the absence of extenuating circumstances such as the incurring 
of a significant operational cash deficit or an economic downturn), an arbitrator may 
determine that termination of the agreements is necessary. In that case, the new 
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agreements detail the Ministry's right to assume uninterrupted operation and control of 
ELRS. A transition period would be required, during which Teranet or its trustee would 
be required to ensure continuity of basic services until a successor service provider was 


established. 


The new agreements also require that a transition plan be developed and approved by a 
joint committee before March 31, 2003. The transition plan is to allow for, if necessary, 
the orderly transfer of responsibility for the operation and control of the basic ELRS 
system and components from Teranet to a new service provider. The plan is required to 
be reviewed annually by the joint committee. 


¢ The Ministry also negotiated improvements in the areas of information privacy, system 
performance and disaster recovery, land registration information quality, liability for 
errors, and control over setting fees for the use of ELRS for basic services. 
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MINISTRY OF PUBLIC SAFETY AND SECURITY 


4.04—Institutional Services and Young 
Offender Operations 


(Follow-up to VFM Section 3.04, Special Report on 
Accountability and Value for Money—2000) 


BACKGROUND 


Since our Special Report on Accountability and Value for Money (2000), the Ministry of 
Correctional Services was consolidated with the Ministry of the Solicitor General in April 
2002 to form the Ministry of Public Safety and Security. Institutional Services and Young 
Offender Operations (Institutional Services), which we audited in 2000, now falls under 
the responsibility of this new Ministry. 


Institutional Services is responsible for the operation of Ontario's correctional institutions. 
These institutions provide custody for adult offenders sentenced to terms of up to two years 
less a day and for accused persons on remand awaiting trial. They also provide custody for 
young offenders between 16 and 17 years of age at the time of their offences. 


For 2001/02, institutional services for adults and young offenders had operating 
expenditures of approximately $500 million. In 1999/2000, operating expenditures were 
about $463 million and there were about 6,200 staff. In the same year, on a daily basis, 


there were approximately 7,400 adult and 700 young offenders in 47 correctional 
institutions. 


In 2000, we concluded that in the process of implementing an infrastructure renewal 
project with estimated capital costs of over $270 million, the Ministry had not properly 
assessed the viability of alternative service delivery options to ensure best value to taxpayers. 
In that regard, we found that: 


¢ The Ministry's decision to finance and build two 1,200-bed correctional institutions at a 


cost of $180 million was not supported by a comprehensive business case assessing the 
risks, costs, and benefits of all relevant alternatives. 


A proper business case and cost-benefit analysis were not done for building a new 
cooking facility within a correctional institution under expansion to provide prepared 
food to a number of correctional institutions. At the completion of our field work, the 
estimated cost to build the facility had increased from $5 million to $9.5 million. As 
well, the facility’s production capacity would be 1,000 meals a day short of meeting the 


needs of the correctional institutions to be served. 


In addition, despite a decline in average daily inmate count (including both adult and 
young offenders) since 1995/96, the operating expenditures for Institutional Services 
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between 1995/96 and 1999/2000 had increased 19% from $388 million to $463 million. 
Specifically, we noted that: 


¢ The number of offenders in the Temporary Absence Program, which is designed to 
protect society by enabling non-violent offenders to maintain community and family 
relationships and responsibilities, had declined from 25,000 to 4,000 between 
1991/92 and 1998/99. The under-utilization of the community program resulted in 
the Ministry foregoing significant potential savings of as much as $50 million a year. 


¢ The average number of sick days per correctional officer increased 38% from 12 days in 
1995 to 16 days in 1998. Overtime expenditures increased 48% from $11.1 million in 
1996/97 to $16.5 million in 1998/99. 


We also found that about 60% of the Ministry’s 47 correctional institutions had security 
non-compliance problems that had not been rectified for up to two years. 


CURRENT STATUS OF RECOMMENDATIONS 


Based on information received from the Ministry of Public Safety and Security, the Ministry 
has taken action to implement the majority of our recommendations, while action is 
planned or in progress for the remaining recommendations. The current status of all our 
recommendations is as follows. 


DUE REGARD FOR ECONOMY AND EFFICIENCY 


Adult Infrastructure Renewal Project 


Recommendation 


When evaluating alternative services delivery options, the Ministry should prepare sound business 
cases, including needs analysis and requirement definitions, to ensure that the option chosen will 
result in best value to the taxpayers. At the minimum, business cases should include an 
assessment of: 


° the costs and benefits of all feasible alternatives; 


° the kind of risks and the level of risks that can be transferred from the province to 
private-sector partners as well as how such risks should be managed: and 


° the proper balance between the objectives of the private sector and the interests of the 
public. 


To ensure realization of anticipated benefits, the Ministry should establish procedures or the 
proper planning and implementation of the chosen service delivery option. 


Current Status 
No other similar projects have been approved since March 31, 2000. 
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Regarding the existing projects, the Ministry has turned over the operating functions of the 
Central North Correctional Centre in Penetanguishene and the Cook-Chill Food Facility in 
Maplehurst to private operators. The Ministry informed us that external consultants were 
hired to ensure the tendering process was fair, open, and transparent. 


Privatization of Young Offender Facilities 


Recommendation 


To ensure proper control over payments to private service providers, the Ministry should more 
closely monitor the operations and billings of private-sector partners that are providing 
alternative services to the province. 


To better ensure future outsourcing provides value for money, the Ministry should complete its 
evaluation of the effectiveness of the pilot project before deciding whether more young offender 


facilities should be turned over to private-sector operators. 


Current Status 


The Ministry indicated that it has implemented a tracking system to be applied to alternate 
service delivery initiatives that continuously monitors operations and billings of private-sector 
partners to ensure they are appropriate. 


With respect to pilot projects involving young offenders, an evaluation report prepared by 
an external consultant in 2002 indicated that Project Turnaround was effective in meeting a 
number of performance measures, including the lowering of the rate of recidivism of young 
offenders. The Ministry is reviewing the young offender capital plans and the results ofa 
supply-and-demand analysis for young offender facilities. Private-sector involvement will be 
reviewed within the context of this review. 


Community Programs 


Recommendation 


To afford non-violent inmates better opportunities for successful reintegration into the 
community and to reduce institutional expenditures, the Ministry should make more effective use 
of community programs. 


Current Status 


The Temporary Absence Program has continued to be underutilized since our last audit 
report. In January 2002, the Ontario Parole and Earned Release Board was given 
responsibility for all unescorted temporary absences lasting 72 hours and longer. 


By February 2002, the Ministry had hired 165 new probation and parole officers and had 


issued a request for proposal to expand the Electronic Surveillance Program so that more 
offenders can be monitored in the community. 
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The Ministry indicated to us that it was developing better programs to help inmates 
reintegrate successfully into the community. However, the changes in the Temporary 
Absence and Electronic Surveillance programs are in the early stages of implementation. 
Therefore, the Ministry is currently unable to assess the full impact of these changes on 
inmates’ successful reintegration into the community and whether institutional expenditures 
have actually been reduced. 


Staffing—Attendance 


Recommendation 


The Ministry should strengthen tts efforts in monitoring sick leave and, where warranted, take 
appropriate corrective action to deal with any problems. 


Current Status 


The Ministry has implemented various guidelines, policies, and procedures to address 
attendance problems. For example, the Human Resources Branch has been staffed with 
four case management co-ordinators for attendance management. A local attendance 
contact position has been established at each correctional institution and probation and 
parole area for the purpose of co-ordinating the monitoring of attendance. 


The Ministry indicated it has reviewed the attendance records of individuals with excessive 
absenteeism and has taken appropriate action. A comprehensive employment tracking/ 
reporting tool was developed in 2001 and reports on absenteeism are now being compiled 
on a quarterly basis. The Ministry reported that the average number of sick days per 
correctional officer decreased from 16 days in 1998 to 14 days in 2001. 


Staffing—Overtime 


Recommendation 
To improve staff attendance and reduce costs, the Ministry should ensure that: 


* actual overtime worked is properly recorded in the attendance management system; 
° overtime ts better tracked and monitored: and 


° staff with regular hours available are used before overtime hours are assigned. 


Current Status 


During 2000 and 2001, the Ministry implemented the Shift Scheduling Administration 
System (SSAS) to create and maintain rotating shift schedules at correctional institutions. 
The scheduling system has now been installed in 24 institutions, 10 of which have been 
using the system since February 2002. The system provides a number of tools to enable 
management at correctional institutions to schedule staff in a manner that more effectively 
manages the allotment of overtime hours. 
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The Ministry has not conducted a formal evaluation of the success of the SSAS in managing 
overtime. According to the Ministry, it has begun developing a method for evaluating the 
SOAS: 


COMPLIANCE WITH LEGISLATIVE 
AND MINISTRY REQUIREMENTS 


Staff Training 


Recommendation 
To establish training programs that help to better protect staff and inmates, the Ministry should: 
° provide up-to-date training to all correctional staff; and 


© maintain current staff training records. 


Current Status 
In 2000/01, the Ministry indicated that it had allocated $5 million in new funding for staff 


training. The Ministry informed us that training for correctional officers and managers is 
continually being developed and delivered. A number of mandatory training hours for 
correctional officers are also being provided. In addition, the Ministry advised us that on an 
ongoing basis new programs are developed to meet the changing needs of correctional 
officers (courses include, for example, Suicide Awareness and Prevention, Mentally-Ill 
Offenders, Effective Correctional Communications, and Stress Management and the 


Correctional Officer). 


The Ministry indicated that it has implemented a system that tracks information regarding 
staff training. This information is updated weekly. 


Correctional Programs 


Recommendation 


To better meet its mandate of protecting society and motivating offenders towards positive 
personal change, the Ministry should ensure that: 


© the correctional needs of offenders are properly assessed and addressed through the 
provision of appropriate programs; and 


the effectiveness of correctional programs 1s evaluated in a timely manner. 


Current Status 


The Ministry informed us that it is ensuring the correctional needs of offenders are properly 
assessed and addressed through the provision of appropriate programs. In adult correctional 
institutions, a comprehensive assessment process is used to identify core offender needs. 
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Core programs based on empirical research are designed to address such needs to reduce 
recidivism. Core programs dealing with anger management, anti-criminal thinking, and 
substance abuse have been developed in order to meet the needs of offenders. 

Since our 2000 audit, the Ministry has established a Program Effectiveness Unit to evaluate 
all core treatment programs and other programs for offenders. The Unit measures both 


process and outcome and provides continuous feedback to operators and senior decision- 
makers on the effectiveness of these programs. 


Protection and Security 
Requirements—Critical Incidents 


Recommendation 


Io reduce or prevent critical incidents, the Ministry should more closely monitor the compliance 
with security measures by its correctional institutions and ensure timely, corrective action in cases 
of non-compliance. 


Current Status 
The Ministry has established various other measures to reduce or prevent critical incidents 


and ensure timely corrective action is taken in cases where security measures are not 


complied with. Such measures include the establishment of: 


* critical incident stress management teams, which educate correctional staff about critical 
incident stress and appropriate intervention measures; 

* institutional security audits; 

® security committees at each correctional institution; and 


* a database for tracking how the recommendations of a Coroner’s jury are responded to 
in order to identify trends and take corrective action to prevent further incidents. 


In addition, the Corrections Accountability Act, 2000 provides for the establishment of a 
local Board of Monitors at Ontario correctional facilities. These boards are to strengthen 
links between the institutions and the local community while ensuring the delivery of 
adequate services. The board members are to act as independent observers, offering an 
objective review of the programs for and care and supervision of offenders. 


Inmates With Mental Disorders 


Recommendation 
To better meet the needs of mentally disordered offenders, the Ministry should expedite its efforts to 


establish treatment facilities and diversion measures for such offenders. 
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Current Status 


The Ministry is in the process of implementing initiatives to restructure correctional 
institutions that provide specialized services for mentally disordered offenders. The Ministry 
entered into a partnership with the Royal Ottawa Hospital to operate a new treatment 
centre in Brockville. According to the Ministry, this two-part project, which is scheduled to 
be completed in 2004, involves both new construction and renovation of existing buildings 
and represents a consolidation of existing beds. As well, the Ministry has committed to 
establishing a 50-bed treatment unit at the new institution in North Bay. However, the 
contract remains to be tendered. 


In spring 2000, the Ministry entered into a partnership with the Centre for Addiction and 
Mental Health at the Maplehurst Correctional Complex for the shared provision of services 
to mentally ill female offenders. Ministry staff and external consultants have started 
developing a document that will outline the programs and services to be offered to female 
offenders. 


The Ministry indicated that it has been working with other justice ministries in developing 
projects for diverting mentally disordered offenders who are inappropriately incarcerated at 
correctional institutions to more appropriate facilities. 


In addition, the Ministry has entered into partnerships with community agencies in Toronto 
to provide case management services. Such partnerships involve case managers working with 
the institution and probation and parole offices to provide assessment, discharge planning, 
and follow-up of mentally disordered offenders. As well, psychiatric nurses are assigned to 
work in conjunction with a Ministry of Health community case planner to deliver 
comprehensive discharge planning services to high-need offenders at one of its jails. 


MEASURING AND REPORTING ON EFFECTIVENESS 


Recommendation 


The Ministry should develop and implement performance measures to assess the effectiveness of 
Institutional Services in motivating offenders towards positive personal change. 


Current Status 


In July 2002, the Ministry implemented a new performance framework for all adult 
correctional institutions, which outlines various outcomes, long-term goals, and key 
performance indicators. The performance framework is to be used to assess compliance with 
ministry standards and the extent to which correctional services, including Institutional 
Services, are achieving their performance objectives. 


According to the Ministry, a performance framework for the young offender and 


community business/operational streams is currently under development, for 
implementation in 2002/03. 
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As well, the Ministry has established re-offending baseline rates. For the first time, the 


province has chosen to use the rate of recidivism (re-offending) to measure performance and 
report to the public. Using a re-offending tracking system, the province will measure the 
effectiveness of correctional policies and programs as it relates to the rehabilitation of 
offenders. Re-offending has been defined as a return to provincial correctional supervision 
on a subsequent conviction. 


eee 


Institutional Services and Young Offender Operations 345 


Follow-up Section 4.04 | 





| Follow-up Section 4.05 





NN 


MINISTRY OF EDUCATION 


4.05—Pupil Transportation Grants to 
School Boards 


(Follow-up to VFM Section 3.05, Special Report on 
Accountability and Value for Money—2000) 


BACKGROUND 


The transportation of students from home to school and back is a major undertaking as 
approximately 800,000 students are transported. The Ministry of Education's 
transportation grants to school boards have increased from $573 million for the 

1999/2000 school year to $610 million for the 2001/02 school year and are expected to be 
about $631 million for the 2002/03 school year. 


In 2000, we concluded that the Ministry, which was working towards its five-year plan to 
revise its funding and accountability relationship with school boards, still had to establish 
satisfactory systems and procedures to ensure that: 


© school boards and the Ministry have the information needed to measure and report on 
transportation service performance; 


° the Ministry has the information needed to establish equitable needs-based funding for 


these services; and 


the costs to provide transportation services to high-needs students are properly tracked 
and managed. 


We made a number of recommendations for improvement and received commitments 
from the Ministry that it would take corrective action. 


CURRENT STATUS OF RECOMMENDATIONS 


The Ministry of Education advised us that it has not fully implemented the 


recommendations made in our Special Report on Accountability and Value for Money 
(2000), although it has made progress on some of them. 


Since our audit, the Ministry has been working to develop the new formula for funding 
student transportation. Until a new funding formula is implemented, transportation grants 
will continue to be based on the 1997 transportation expenditures of school boards, with 
adjustments for changes in enrolment or extraordinary cost increases. Consequently, the 
funding inequities that existed in 2000 and their impact on the levels of service provided 
remained in place at the time of this follow-up review. 
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The current status of each of our recommendations is outlined below. 


EFFORTS TO REDUCE TRANSPORTATION COSTS 


Recommendation 


To help ensure that school boards implement best practices to deliver transportation services as 
cost effectively as possible, the Ministry should: 


* expedite the development of the new funding formula; 


* ensure that the new funding formula contains incentives for boards to implement cost- 
saving measures; and 


° establish target dates for school boards that have not already done so to either form 
transportation consortia or submit a business case demonstrating why it would not be 
beneficial for them to integrate the delivery of their transportation services. 


Current Status 


Although adjustments have been made to the funding allocations for individual school 
boards, the Ministry continues to provide funding based on the funding formula in use in 
2000 and does not expect to have a new formula in place until the 2003/04 school year. 


The Ministry has been working with a committee that includes school board and school-bus 
operator personnel to develop a funding formula proposal. The proposed formula involves 
using transportation-management software qualified by the Ministry to calculate the 
transportation needs of each board based upon standard service parameters and assuming 
that boards serving the same area form consortia to jointly deliver transportation services for 
their students. Such a formula would provide an incentive for boards to form consortia, thus 
eliminating the need for the Ministry to require that boards prepare business cases or meet 
target dates as per our recommendation in 2000. 


The proposed formula was tested at three boards and five consortia (involving 11 boards). 
Due to data limitations, such as incomplete student data and digital maps, the Ministry was 
not able to use the results of these tests to finalize the details of the funding formula. 
However, the test results did assist the Ministry in identifying problem areas that need to be 
addressed, and this information will be incorporated into a strategy paper that is to be 
completed in the fall of 2002. 


INFORMATION FOR DECISION-MAKING 


Recommendation 
Io facilitate prudent decision-making and strengthen local accountability, the Ministry should: 
° ensure that boards acquire and utilize information systems that can provide the 


information needed to enable trustees to make informed decisions about the level of 
transportation services offered; and 





Pupil Transportation Grants to School Boards 347 


| Follow-up Section 4.05 | 








| Follow-up Section 4.05 | 


<< 


° require boards to develop reports that relate costs to level of service in order to ensure 
that the Ministry and trustees can compare boards to each other and to established 
benchmarks. 


Current Status 

The Ministry does not intend to require boards to develop reports that relate costs to the 
level of service provided. However, the Ministry has taken the following significant steps to 
ensure that boards have the ability to provide the Ministry and trustees with this 


information: 


© The Ministry has facilitated the acquisition of appropriate student transportation 
management software by: 


- managing a request-for-qualification process to evaluate various software products 
and ensure that those that the Ministry qualified can design cost-effective bus routes 
and meet the information needs of school boards and the Ministry (five products 
were qualified for use by school boards); and 


- establishing a $7-million loan program to support school boards in the purchase of 
qualified software products (at the time of our follow-up review, 66 of the province's 
72 school boards had acquired such software). 


© The Ministry required that boards complete a survey on key transportation operational 
data for the 2000/01 school year. Summarized results were posted on a ministry Web 
site, thus enabling boards to compare their costs and their levels of service with those of 
other boards. A second survey, revised from the first one to collect more detailed 
information for the purpose of developing indicators to assist trustees and boards to 
make informed business decisions, was issued in May 2002. School boards have been 
advised that completion of this survey will be an annual requirement. 


CLARIFYING LEGAL AND POLICY REQUIREMENTS 


Recommendation 


To help ensure that school boards provide transportation services that meet local needs in an 
equitable manner and adhere to policy and legal requirements, the Ministry should: 


© provide guidance to boards on the spending and service level information that they 


should publish in order to facilitate informed public debate among trustees and their 
community; and 


° clarify the legal and policy parameters that it expects boards to operate within. 


Current Status 


The Ministry does not intend to provide guidance to school boards on the information that 
they should publish about their own transportation services. However, the Ministry has 
developed a Web site that contains: policy memoranda; the results of the transportation 
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surveys (including summary reports regarding spending and service levels); and links to 


other government Web sites to provide school boards with easy access to regulatory 
requirements for school bus operation. Currently, the Ministry’s Web site is available only to 
school boards and school bus operators. The Ministry intends to grant public access to the 
Web site in the future, which would enable the general public to compare transportation 
services and costs among school boards. 


The Ministry clarifies legal and policy issues as the need arises. For instance, in April 2002, 
the Ministry sent a letter to all school boards that clarified its policy with respect to 
prohibiting school boards from charging fees for transportation services. 


TRANSPORTATION OF SPECIAL NEEDS STUDENTS 


Recommendation 


To enhance accountability and to ensure equitable funding of transportation for special needs 
students, the Ministry should: 


° require boards to track and report these costs separately; and 


° consider funding transportation costs for high needs students through the Special 
Education Grant. 


Current Status 

The first part of the recommendation was implemented, insofar as the Ministry will require 
that boards track the cost of transportation for students with special needs in order to 
complete the transportation survey that the Ministry is intending to annually require from 
boards. 


The second part of the recommendation was not implemented. Currently, special 
education program managers lack the information needed to make economical decisions 
about whether to provide a given service at local schools or instead provide it on a more 
centralized basis and transport students to the service location. They also lack the incentive 
to do so, because neither alternative impacts the special education budget. 


FUNDING POLICY FOR SAFETY HAZARDS 


Recommendation 


To better ensure that transportation grants are equitable and utilized in a cost-effective manner, 
the Ministry should: 


° clearly specify the conditions, if any, under which it is appropriate for school boards to 
use transportation grants to address municipal responsibilities; and 

° ensure that the funding mechanism provides incenti ves for school boards and 
municipalities to work together to implement the most economical solutions for 
addressing safety hazards. 
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Current Status 

The Ministry is still examining options on how best to deal with this issue. In cases where 
municipalities have not taken appropriate corrective action regarding hazards, the Ministry 
continues to provide boards with the funding required to provide additional bus service. As 
a result, the province continues to fund bus services in cases where alternative solutions to 
safety hazards exist that are more economical and municipally funded. 





350 2002 Annual Report of the Provincial Auditor of Ontario 


eee 
MINISTRY OF THE ENVIRONMENT 


4.06—Operations Division 


(Follow-up to VFM Section 3.06, Special Report on 
Accountability and Value for Money—2000) 


LL 
BACKGROUND 


The Operations Division is responsible for administering the Ministry’s approvals and 
enforcement activities. It also responds to reports of pollution and spills that may have health 
and environmental impacts. In addition, the Division cleans up abandoned contaminated 
sites using funds available in the Environmental Clean-Up Fund. 


For the 2001/02 fiscal year, the Division had total expenditures of $83 million and about 
895 staff. For the 1999/2000 fiscal year, it had total expenditures of $62 million and about 
680 staff. 


In our Special Report on Accountability and Value for Money (2000), we concluded that the 
Ministry did not have satisfactory systems and procedures in place to administer approvals 
and to enforce compliance with environmental legislation. Our major concerns included: 


* Certificates of approval are required for any facility that discharges contaminants into 
the environment. The Ministry's systems did not enable it to assess whether the facilities 
for which over 220,000 certificates of approval had been issued since 1957 were in 
compliance with current environmental standards. As a result, the Ministry did not 
know the extent to which facilities were not meeting current environmental standards 
and, consequently, where corrective action had to be taken. 


* An estimated over $90 million in financial assurance was not obtained from facility 
operators, as required under legislation, to clean up potential damages that operators 
may cause to the environment. 


* A 25% reduction in regional staff over the four-year period from 1995/96 to 
1999/2000 had contributed to a 34% decrease in the number of ministry-initiated 
inspections conducted per year. Further, the Ministry relied extensively on facility 
operators to comply voluntarily rather than impose available stringent enforcement 
measures. This was of particular concern as one-third of violations found by ministry 
inspectors were repeat violations and the Ministry identified significant violations in 
31% of the inspections it conducted. 


* The Ministry usually learned of contaminated sites only after serious harm to the 
environment had already occurred. 
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© Over $10 million in fines had accumulated over many years. The Ministry had not 
supported the collection of fines as it could have by aggressively enforcing 
environmental legislation that allows it to suspend violators’ operations if necessary. 


We also concluded that the Ministry was not measuring and reporting on its performance 
in a comprehensive and objective manner in order to demonstrate its progress in managing 
the environment. 


Accordingly, we made a number of recommendations for improvement and received 
commitments from the Ministry that it would take corrective action. 


CURRENT STATUS OF RECOMMENDATIONS 


Based on information received from the Ministry, we concluded that continuing action was 
required to implement most of our recommendations. The current status of each of our 
recommendations is outlined below. 


COMPLIANCE WITH ENVIRONMENTAL LEGISLATION 


Certificates of Approval—Updating Certificates of 
Approval 


Recommendation 


In order to ensure that existing certificates of. approval reflect current environmental standards, 
where required, and are being applied consistently, the Ministry should: 


° improve its information systems so that all certificates of approval can be assessed on the 
extent to which they need to be updated with new conditions and requirements; 


develop systems that would allow for updating certificates of approval in a timely and 
efficient manner; and 


establish action plans and timetables for when certificates of approval will be required 
to be up-to-date. 


Current Status 


The Ministry was continuing its efforts to implement a new, computerized Integrated 


Divisional System (IDS), which was expected to improve the informational needs and 
efficiency of the Division. 


In November 1999, the Ministry began using IDS to record and track certificates of 
approval. As of March 31, 2002, approximately 115,000 certificates previously stored 


within various databases and on a manual card index system were now recorded on IDS. 


Since August 2000, the Ministry has updated approximately 3,000 certificates of approval 
for certain higher-priority facilities, such as water treatment plants serving municipalities. In 
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addition, the Ministry completed its internal program effectiveness review of its certificate- 
of-approval program in July 2001. The program effectiveness review resulted in 10 
recommendations for updating certificates of approval to reflect current environmental 
standards, including a recommendation that all certificates of approval undergo a 


mandatory review every five to 10 years. 


In January 2002, the Ministry’s preliminary analysis on the extent to which certificates of 


approval require updating was as follows. 


Ministry Analysis of Certificate-of-approval Updating Needs 









Extent of | eaitaniee 

Updating Reason : te 

Required Certificates | 
: Affected _ 





No significant change in environmental requirements has 
occurred (e.g., for watermains and storm and sanitary sewers) 


Certificate holder no longer in operation 


None 












120,000 








N/A 












New exemptions in effect (e.g., for food preparation 
equipment) 





None 








Unknown Insufficient information available 





Source of data: Ministry of the Environment 


The Ministry also indicated that it has developed a three-point pilot project that would 
support preparation of a comprehensive strategy for updating certificates of approval. The 
strategy includes: 


* requiring updates for certificates of approval in cases where the certificate holder is 
applying to make significant changes to its operations; 


* implementing a field-alert system to support compliance and inspection activities for 
identifying certificate holders whose certificates of approval require updating; and 


* _arisk-based proactive identification of strategically selected sectors in which certificates 
of approval require updating. 


The three-point pilot project is expected to result in the updating of about 1,300 certificates 
of approval during the 2002/03 fiscal year. 


During the summer of 2002, the Ministry was expecting to refine the estimates of the 
number of certificates of approval requiring updating. Based on the refined estimates, 
experience from pilot projects, and a consideration of the recommendations resulting from 
the program effectiveness review, a strategy for updating existing certificates of approval is to 
be developed over the next fiscal year. The plan and timetable as to how and when the over 
100,000 remaining certificates of approval may be recorded on IDS is also to be established 
as part of the new strategy. 
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Certificates of Approval—Financial Assurance 


Recommendation 


To minimize financial risk to the province relating to environmental clean-up costs, the 


Ministry should: 


© establish controls to ensure financial assurance requirements are assessed and specified 


in the certificates of approval and then followed; and 


° identify, for each facility operator, the correct amount of financial assurance 
outstanding and take timely action to obtain the necessary assurance. 


Current Status 


The Ministry has completed a review of internal procedures for financial assurance and has 
established and incorporated the results into a revised draft guideline. The revised draft 
ouideline updates criteria as to when financial assurance should be required, how the 
amount of financial assurance should be determined, what form of financial assurance 
should be provided, and when financial assurance should be used by the Ministry to pay for 


compliance activities. 


In addition, the financial assurance database has been upgraded. A user’s manual and an 
administrator’s manual have been completed for use by staff. As ministry staff access the 
database, they are also reviewing existing data to confirm its validity. In addition, a specific 
project is underway to double-check data validity and update the information. 


The Ministry has validated previously identified potentially delinquent accounts and has 
determined that 155 of the original 553 certificate-of-approval holders were in fact 
delinquent in meeting financial assurance requirements. The Ministry indicated that, as of 
March 31, 2002, its follow-up had resulted in receiving confirmation that 69 operations 
were no longer in business, and compliance was obtained from 53 operations. The Ministry 
was continuing its efforts to collect from the remaining 33 delinquent accounts, and regular 
progress reports were being submitted to senior ministry management. As a result, about 


80% of outstanding financial assurance cases have been resolved, and approximately 
$9 million has been collected. 


Certificates of Approval—Costs Of Monitoring 
Compliance 


Recommendation 


To enable the Ministry to cost-effectively increase the scope of its environmental oversight role, tt 
should consider imposing conditions on certificates of approval that include: 


° greater use of self-reporting requirements that demonstrate owner compliance, 
including, where warranted and practical, certification by independent experts; and 


owners paying a greater share of costs associated with monitoring their compliance. 
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Current Status 


In 2000/01, the government passed new regulations requiring that self-reporting and 
detailed third-party engineering assessments for municipal water treatment plants be 
submitted to the Ministry every three years. The reports and assessments are to be checked 
for compliance with the plants’ certificates of approval and current drinking-water 
standards. 


In addition, the Ministry has established or expanded a few pilot projects with several large 
operators with proven track records to test new approaches and best practices that include 
self-reporting and owners paying a greater share of the costs of monitoring their 
compliance. 


However, the Ministry has not developed the guidelines indicated in its response to our 
recommendation, which would identify appropriate activities, circumstances, and 
mechanisms that require third-party compliance verification by independent experts and 
reporting. 


Enforcement—Iinspection Coverage 


Recommendation 


lo more effectively enforce compliance with environmental leeislation, the Ministry should 
yy BF ay 
explore options and develop procedures for significantly increasing its inspection coverage. 


Current Status 


The Ministry has introduced several changes to increase its inspection coverage. Most 
notably, in September 2000, the Ministry introduced an Environmental SWAT Team, 
which focuses on priority areas where compliance by industries or companies is a major 
concern. The team includes approximately 30 dedicated inspectors capable of conducting 
approximately 1,000 inspections per year. 


In January 2001, the Ministry hired an additional 25 inspectors to allow it to meet its 
commitment of conducting annual inspections of all municipal water treatment plants 
across the province. 


The following chart shows the number of inspections over the last three years. 


Number of Inspections, 1999/2000-2001/02 


1999/2000 
2000/01 
2001/02 











Source of data: Ministry of the Environment 
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In June 2002, the Standing Committee on Public Accounts issued its report on hearings 
held on our audit report with representation from the Ministry. The Committee 
acknowledged the Ministry’s recent commitment to improve its enforcement but concluded 
that a level of assurance was required from the Ministry to guarantee the inspection 
commitment over the long term. As a result, the Committee made the following 
recommendation: “The Ministry of the Environment should assess its risk, review, and 
inspection coverage for the period 2001-2005 and report to the Standing Committee on 
Public Accounts by June 1, 2003 on its corrective action to improve inspection coverage to 
reduce the range of environmental risks.” 


Enforcement—Resolving Violations 


Recommendation 


To make enforcement actions more timely and effective, the Ministry needs to strengthen its 
enforcement activities by: 


© taking appropriate action on violations and following up on a more timely basis; and 


° ensuring policies and procedure manuals encourage the use of more stringent 
compliance measures, where appropriate. 


Current Status 


In March 2000, the Ministry issued a directive to district staff providing clarity and 
direction on mandatory abatement and instructing staff to strictly adhere to the existing 
compliance policy. In addition, monthly abatement/enforcement activity reports on the use 
of enforcement tools available under the legislation to compel corrective action were 
introduced. According to the Ministry, this has resulted in more aggressive use of 
mandatory abatement actions against violators, including issuing control orders that specify 
actions and completion dates to bring about compliance. For example, the total number of 
control orders issued by environmental officers to operators requiring corrective action in 
1999/2000 was less than 500; in 2000/01, almost 1,500 orders were issued; and in 
2001/02, over 2,200 were issued. 


In addition, the Ministry anticipates that, by fall 2002, it will be in a position to commence 
imposing new administrative monetary penalties on violators of up to $10,000. The use of 
administrative monetary penalties is expected to significantly increase and strengthen the 
range of enforcement tools available to the Ministry for obtaining compliance. 


Enforcement—Significant Versus Minor Violations 


Recommendation 


To minimize environmental and health risks, the Ministry should: 


reassess its policies, procedures, and criteria for determining the severity of violations; 


and 
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* ensure that the significance of preventative measures is better understood and 
communicated to staff. 


Current Status 


The Ministry indicated that it has reviewed its operating policy and procedures to ensure 
that the severity of violations is consistently and appropriately assessed and that the full range 
of compliance tools is used across all programs. In addition, a revised draft compliance 
policy with strengthened considerations of human health and environmental consequences 
has been developed. This draft is expected to be finalized in the 2002/03 fiscal year. The 
revisions are largely based on the experience obtained through the SWAT risk-compliance 
project and take into account the implementation of administrative monetary penalties. 


Enforcement—Managing Inspections 


Recommendation 


1o make its inspection program more effective in supporting the enforcement of environmental 


legislation, the Mi inistry should: 


° review its policies, procedures, and guidelines over ministry-initiated inspection 
activities to ensure that adequate record-keeping and reporting requirements are in 


place; and 


* ensure that inspections are consistently planned for and conducted. 


Current Status 


As an interim measure, the Ministry implemented two databases for the collection of 
information associated with planned inspections: the Interim Inspection System for water 
treatment facilities was introduced in 2000, and the Municipal Sewage Inspection System 
was introduced in 2001. In order to enhance and maximize its inspection planning, the 
Ministry has, in several program areas, instituted centralized planning of, resource allocation 
for, training in, tracking of, and evaluation of inspection activities. 


In addition, the Ministry indicated that it has reviewed and amended its annual workplan 
for inspection priorities and frequency for the various types of operations to ensure that 
regional offices are addressing all inspection priorities. 


The Integrated Divisional System operational plan and inspections components were 
scheduled for rollout to ministry staff in fall 2002. The operational plan and inspections 
components are expected to help staff plan, conduct, and record inspections on a consistent 
and priority-driven basis. 
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Enforcement—Management Information 


Recommendation 


To better support and to improve the delivery of its enforcement efforts n protecting the 
environment, the Ministry should: 


© establish a system to identify all conditions of approvals that require follow-up by 
specific dates to assess owner compliance and, where necessary, initiate timely 
enforcement action; 


¢ develop an accurate and comprehensive management information system to assist in 
identifying and prioritizing facilities for inspection; and 


¢ periodically assess the types and frequency of violations and the effectiveness of 


enforcement measures used. 


Current Status 


The new Integrated Divisional System was designed specifically to support and improve the 
information needed for enforcement. According to the Ministry, the development of the 
remainder of the system, including modules for pollution incident reporting, inspection 
activities, investigation/ prosecution, operational planning, and time and task tracking, was 
completed in September 2001. In addition, a module to assist with the follow-up and 
tracking of conditions on certificates of approval was completed in November 2001, and a 
field-alert prototype system to allow field staff to request an update or revocation of any 
environmentally significant, out-of-date certificates of approval was developed in December 
2001. Primary testing, system debugging, and staff training were in progress, and the 


operational plan and inspections components were scheduled for rollout to ministry staff in 
fall 2002. 


When fully functional, the system is expected to run reports focused on facility type, 
violation type, and enforcement action, as well as inspection reports. This will allow for 
strategically targeting compliance inspection and enforcement activities and would focus 
field staff efforts on areas that have the most impact on environmental protection. 


Enforcement—Unpaid Fines 


Recommendation 


To be more effective in enforcing environmental legislation and to improve collection of 
outstanding fines, the Ministry should investigate the reasons why outstanding fines are unpaid 


and use its statutory authority to suspend environmental approvals for violators who do not pay 
their fines. 


a 


358 2002 Annual Report of the Provincial Auditor of Ontario 


eee 
Current Status 


The process of issuing and collecting environmental fines is a shared responsibility between 
the provincial and municipal governments. The Ministry is responsible for conducting the 
majority of the environmental prosecutions in the province; the municipalities have 
responsibility for collection of the fines, and the Ministry of the Attorney General and the 
municipalities keep the fines information current on the provincial Integrated Court 
Offences Network (ICON) tracking system. The Ministry is now using ICON data as the 


source data to track and monitor information related to unpaid environmental fines. 


According to the Ministry, fines assessed for the period from 1992 to December 31, 2001 
totalled $26.9 million, of which $15.9 million has been collected, leaving an outstanding 
balance of $11 million. Ministry analysis indicates at least $2.7 million may not be 


collectable due to bankruptcy and insolvency, which leaves an outstanding balance of 
$8.3 million. 


The Ministry was revising its business practices to support the ongoing collection of unpaid 
fines. The Ministry plans to regularly review the information related to unpaid 
environmental fines and identify those defaulters who hold an environmental permit, 
licence, or certificate of approval. The Ministry indicated that it will contact the appropriate 
municipalities and advise them that it is prepared to support municipal collection of fines 
using suspensions until payment is received. In addition, a new procedure in the Ministry’s 
approval process is to be instituted in which all applicants are to be screened against unpaid 
fines. This process is to be in place before March 31, 2003. 


Contaminated Sites—identifying Contaminated Sites/ 
Liability and Financial Considerations 


Recommendation 


To help ensure that the Ministry’ efforts and programs are effective in minimizing damage to the 
environment, the Ministry should develop a strategy for: 


° the early identification of all contaminated sites and responsible parties, which would 
allow for prioritizing clean-up requirements; and 


° the establishment of incentives to encourage property owners to voluntarily clean up 
contaminated sites. 


Current Status 


In response to our Special Report on Accountability and Value for Money—2000, the 
Ministry informed us that it would consider developing a strategy, with the co-operative 
involvement of other levels of government, to expand efforts that would encompass a wider 
range of contaminated sites and address means of identifying those sites earlier and 
encouraging clean-up of those sites. 
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The Ministry indicated that it is a key contributor to the multi-ministry strategy to facilitate 
the clean-up and redevelopment of contaminated sites. The contaminated sites include 
brownfield sites, which typically are large, abandoned, or decaying industrial or commercial 
sites with contamination that limits their future use. The Brownfields Statute Law 
Amendment Act, 2001 received Royal Assent on November 2, 2001 and provides incentives 
for cleaning up contaminated sites, as well as mechanisms that ensure that sites meet 
appropriate standards when an owner requests a change in use. This legislation establishes: 


© rules and requirements for the assessment, clean-up, and limitation of environmental 
liability for contaminated sites; 


¢ mandatory filing of a record of site condition in a publicly accessible environmental site 
registry; and 


¢ changes that streamline the planning process to expedite brownfield projects and allow 
municipalities to provide financial support for site assessment and clean-up costs 
associated with contaminated brownfield sites. 


The Ministry was in the process of developing regulations for implementing the legislation 
and was developing the site registry, with targeted completion by fall 2002. 


In addition, from August 2001 to March 2002, the Ministry provided $10 million in 
funding for groundwater studies to help municipalities map sensitive groundwater areas 
and to identify potential risks, such as contaminates, to groundwater sources. 


However, the Ministry has not initiated significant changes to its operations to deal with the 
other risks of site contamination that we identified in 2000; such changes would allow for 
the timely identification of all contaminated sites and the parties responsible for their clean- 
up. For instance, no new measures were introduced to: 


¢ identify existing industrial and commercial sites that were a high risk for contamination; 


¢ determine whether closed private and municipal landfill sites posed a hazard to 
surrounding areas; and 


° identify abandoned underground fuel storage tanks, which are the focus of about half 


of the projects being funded by the Environmental Clean-Up Fund administered by 
the Ministry. 


Information System Contract 


Recommendation 


1o facilitate the efficient and effective delivery of the Division’s programs, the M; inistry should 
ensure that the remaining portions of the Integrated Divisional System are completed in a timely 
and cost-effective manner and in accordance with required approvals. 
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Current Status 


On February 15, 2000, the Management Board of Cabinet (MBC) approved a total cost of 


about $2.9 million for this project, which included actual and estimated costs from the 
1997/98 fiscal year to the 2002/03 fiscal year. As of March 31, 2002, the Ministry had 
spent the entire amount approved by MBC. Certain components of the system were 
already in use, and others, while further delayed, are scheduled to be operational during the 
2002/03 fiscal year, following final testing and training of staff. 


MEASURING AND REPORTING ON PROGRAM 
EFFECTIVENESS 


State of the Environment Reporting 


Recommendation 


Lo ensure that the Ministry’s progress in managing the environment is measured and 
communicated tn an objective manner, a state of the environment report should periodically be 
Prepared using a set of comprehensive outcome measures that assess the quality of and changes to 
the environment over time. 


Current Status 


The Ministry has neither prepared nor committed to prepare a state of the environment 
report. Instead, the Ministry has continued with its long-term strategy of using its Web site 
to publicly share information on its own activities, such as results of enforcement efforts, as 
well as information on how the higher-risk regulated sectors (for example, municipal water 
works and the manufacturing and mining industries) are performing in their responsibilities 
related to the environment. 


Divisional Performance Reporting 


Recommendation 


10 provide a more comprehensive assessment of the Division's contribution to protecting the 
environment, the Ministry should develop more results-oriented performance indicators to 
measure and report on the effectiveness of the Division’s operations. 


Current Status 


The Ministry was continuing to use its existing activity-based performance measures but 
indicated it was working towards the development of outcome-based performance 
measures. [he Ministry expects that it will be better able to generate the data needed to 
develop enhanced performance measures when the new information systems in progress 
become operational. 
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MINISTRY OF FINANCE 


4.07—-Retail Sales Tax Program 


(Follow-up to VFM Section 3.07, Special Report on 
Accountability and Value for Money—2000) 





BACKGROUND 


The Retail Sales Tax Act currently imposes a general sales tax of 8% on the retail price of 
most goods and services sold to final consumers. Various tax exemptions cover thousands of 
items and are aimed at reducing tax regression or promoting economic or social objectives. 
Examples of such exemptions include children’s clothing, equipment for use by people with 
disabilities, and goods purchased by Status Indians under certain conditions. 


At December 31, 1999, approximately 380,000 vendors were registered to collect and 
remit retail sales tax (RST) to the province. RST receipts for the 1999/2000 fiscal year 
totalled approximately $12.6 billion, net of $159.5 million in refunds, and represented 
21% of the province’s total revenue. For the 2001/02 fiscal year, RST receipts totalled 
approximately $13.8 billion and represented 28.6% of the province's total taxation revenue. 


Although RST revenues had increased significantly as of 1999/2000, and the Ministry had 
made improvements to its administration of this program since we last audited it in 1995, 
we concluded that the Ministry needed to further improve its procedures because it did 
not: 


* conduct research into the underground economy in order to identify sectors of the 
economy in need of more rigorous compliance and enforcement action; 


¢ have adequate procedures in place to ensure that all vendors that should have been 
registered with the Ministry to collect RST were in fact registered; 


* include all segments of the small vendor population in its audit coverage; 


select vendors for audit more representatively with a view to encouraging broad-based 
voluntary compliance; 


follow up on all overdue vendor sales-tax returns on a timely basis; and 


often make adequate or timely collection efforts, particularly for the many smaller 
outstanding balances. 


We made a number of recommendations for improvement and received commitments 
from the Ministry that it would take corrective action. 
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CURRENT STATUS OF RECOMMENDATIONS 


The Ministry had taken or was in the process of taking corrective action on all of our 
recommendations. The status of each of our recommendations as of June 30, 2002 is 


detailed in the following. 


TAX ROLL 


Recommendation 


1o help ensure that the tax roll for vendors that sell taxable goods and services is complete and 
accurate, the Ministry should: 


* work with the Ministry of Consumer and Commercial Relations [which has since 
become the Ministry of Consumer and Business Services] to ensure that it is appropri- 
ately advised of newly registered businesses for potential inclusion in the tax roll: and 


° implement additional procedures needed for identifying non-registered vendors selling 
taxable goods and services through data matching with other government databases or 
at their places of business. 


Current Status 


The Ministry advised us that information technology business requirements and programs 
are being developed to link the retail sales tax and corporations tax databases (corporate 
registrations received from the Ministry of Consumer and Business Services) during the 
2002/03 fiscal year. In addition, initial contact with the Ministry of Consumer and Business 
Services has been established to explore opportunities to match the business names registry 
with the RST database. Details are being worked out, and the Ministry hopes to obtain the 
necessary data during the 2002/03 fiscal year. 


The Ministry has matched the RST and the federal GST databases for certain high-risk 
sectors and advised us that the number of non-registrants identified was somewhat less than 
originally anticipated. The Ministry is awaiting the results from matching other business 
sectors. 


OVERDUE SALES TAX RETURNS 


Recommendation 


To identify and follow up on outstanding retail sales tax returns and required remittances more 
promptly, the Ministry should ensure that: 


¢ vendors with overdue tax returns are contacted within the required time periods; and 


¢ when necessary, estimated assessments are issued on a more timely basis. 
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Current Status 

In May 2001, the Ministry received approval from Management Board of Cabinet to 
temporarily double staffing in this area to address the backlog. The extra staff were on board 
and fully trained as of December 2001, at which time the Ministry anticipated that the 
backlog would be eliminated within two years. The Ministry ts currently assessing the 
impact of the labour disruption in the spring of 2002 on its goal of eliminating the backlog. 


COLLECTIONS 


Recommendation 


To help maximize the collection of all outstanding accounts receivable, the Ministry should 
ensure that all accounts receive timely action by a collector, including those under $10,000. 


Current Status 


The Ministry has improved the management information available in its information 
technology system in several important respects and is developing the necessary reports to 
effectively monitor the level of activity at all stages of the collection function. The Ministry 
also intends to conduct a series of operational reviews of the collection function 
commencing in September 2002. 


ENFORCEMENT 
Tax Gap 


Recommendation 


To help reduce the tax gap, the Ministry should conduct the research necessary to identify 


significant aspects of the underground economy and focus its compliance and enforcement efforts 
on these aspects. 


Current Status 


The Compliance Planning and Research Unit referred to in the Ministry’s response to our 


recommendation was established in February 2001, and all staff were on board by 
October 2001. 


The Ministry is part of the National Working Group on the Underground Economy. Joint 
compliance research at a national level is expected to move forward over the next year along 
with several pilot initiatives in the areas of data sharing, enforcement, and education. We 
were also informed that the Ministry regularly exchanges data and information on audit 
assessments and non-registrants with the Canada Customs and Revenue Agency. 
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Audit Coverage 


Recommendation 


16 detect unpaid retail sales taxes owed to the province and to encourage broad-based voluntary 
compliance, the Ministry should include all segments of the small vendor population in its audit 
coverage and attain overall audit coverage levels that will ensure visibility in the broader vendor 
community. 


Current Status 

In October 2001, the Ministry completed a detailed review of small vendor audit coverage 
by profile code to identify areas requiring greater audit visibility to achieve balanced 
coverage. We were advised that the results of the review were discussed by the appropriate 
senior managers and that a plan of action to increase small vendor audit coverage was 


developed for implementation in the 2002/03 fiscal year. 


Audit Selection 


Recommendation 
To help ensure that it meets its objectives for encouraging voluntary vendor compliance as well as 


for recovery of retail sales taxes owing, the Ministry should: 


° select vendor files for audit on a more representative basis, with a view to encouraging 
broad-based, voluntary compliance; and 


° ensure that selected audits commence and are completed on a timely basis. 


Current Status 


The Ministry has commenced a review of its audit selection approach and strategy in 
consultation with the regional tax offices. We were also advised that the development of a 
centralized divisional audit selection system is under review. 


In addition, joint committees of staff from the Canada Customs and Revenue Agency and 
the Ministry now meet quarterly to better co-ordinate their respective audit efforts and 
share audit-related information. 


The Ministry also has established timelines for the completion of audits and has developed 
quarterly management reports to monitor compliance with these requirements, effective 
October 31, 2001. However, a ministry review found that timely completion of audits 
continues to be a concern in some cases. 


eae a 8 Oe 
Retail Sales Tax Program 365 


Follow-up Section 4.07 | 








| Follow-up Section 4.07 | 


ES 


Tax Refunds 


Recommendation 


To help ensure that only eligible refunds of retail sales tax are made, the Ministry should 
maintain a list of refunds issued subject to audit and ensure that, where refunds are issued subject 
to audit, the necessary follow-up work is performed. 


Current Status 


A quarterly report on refunds processed subject to audit is regularly being provided to the 
appropriate offices for monitoring purposes. 


Penalties 


Recommendation 
To provide an effective deterrent, the Ministry should consider imposing the legislated 25% 


penalty in cases where correct retail sales tax amounts are not remitted as a result of taxpayer 
neglect, carelessness, willful default or fraud. When a penalty is not imposed, the Ministry should 


ensure that the reasons are clearly documented. 


In addition, the Ministry should review and determine whether the threshold amount below 
which penalties are not assessed is appropriate. 


Current Status 


We were advised that all offices were instructed to adhere to the existing policy. 


The Ministry is also in the process of reviewing the current threshold amount under which 
penalties are not imposed. A draft briefing paper outlining a revised policy for the 25% 
penalty is currently being reviewed. Upon approval of the briefing paper, the Ministry 
intends to make appropriate revisions to the Audit Handbook regarding the 25% penalty. 
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MINISTRY OF HEALTH AND LONG-TERM CARE 


4.08—-Community Health Centre Program 


(Follow-up to VFM Section 3.08, Special Report on 
Accountability and Value for Money—2000) 


——— 
BACKGROUND 


Community health centres (CHCs) provide health care, health promotion, and other 
health, educational, and social services to identified ptiority groups within their geographical 
areas. Unlike most primary health care providers, which are funded on a fee-for-service 
basis, CHCs have fixed budgets and provide services using salaried staff. For the 2001/02 
fiscal year, the Ministry provided approximately $117 million to fund CHCs. In 
1999/2000, it provided approximately $87 million. 


In our 2000 audit, our major concerns with the Program were: 


* The Ministry had not assessed the efficiency, effectiveness, and ability of CHCs to 
provide quality care. 


* Funding for CHCs was not linked to the expected amount of services to be provided, 


the number of clients to be served, or the anticipated outcomes. 


* The Ministry had no assurance that CHCs regularly review the quality of care they 


provide and the services they deliver. 


We made a number of recommendations for improvement and received commitments 
from the Ministry that it would take corrective action. 


CURRENT STATUS OF RECOMMENDATIONS 


According to information we received from the Ministry of Health and Long-Term Care, 
the Ministry has taken some action on all of the recommendations we made in our Special 
Report on Accountability and Value for Money (2000). The current status of each of our 


recommendations is outlined below. 


CHCS AND PRIMARY CARE REFORM 


Recommendation 

16 ensure that any additional investments in CHC are justified, the M. inistry should first 
evaluate the efficiency and effectiveness of CHCs in providing quality primary health care and 
compare the results with other models of primary care delivery: 
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Current Status 


According to the Ministry, a strategic review of the CHC Program began in December 
2000. A steering committee comprising various internal and external stakeholders issued a 
final report on June 8, 2001, which is now available to the public on the ministry’s Web site. 
The Ministry developed implementation options in response to the results of the strategic 
review, and an action plan was approved for inclusion in the Ministry's 2002/03 business 
planning allocation process. The Ministry has begun work to identify alignment issues for 
integrating the CHC Program more fully with the objectives of primary-care reform. 


The Ministry indicated that, while CHCs have been collecting information, there were 
significant problems with the new data extraction software that was implemented in 
October 2000, therefore limiting the Ministry’s ability to utilize the information. A new 
version of the software, based on input from the field, was implemented in early 2001. 


CHCs were surveyed in July 2001 to determine the extent of outstanding data quality 
issues. In response, a phased approach to transmitting data from the CHCs to the Ministry 
has been implemented. In January 2002, the Ministry updated its management 
information system, which improved the system's usability. 


The Ministry advised us that it would continue to evaluate the experiences of other 
jurisdictions to identify best practices and approaches that warrant consideration in Ontario. 


FINANCIAL MANAGEMENT 


Recommendation 


To better enable it to assess whether CHCs use their funding economically and in accordance 
with funding arrangements, the Ministry should: 


° ensure the timely receipt of audited financial statements and activity reports; and 


° monitor CHC expenditures during the year and adjust cash flows where warranted. 


Current Status 


The Ministry advised us that a revised system to ensure the timely receipt of audited 
financial statements and activity reports was put in place in 2002. 


The Ministry informed us that expenditures for the 2001/02 fiscal year have been reviewed 


and cash flows adjusted where warranted. 


ASSESSING CHC PERFORMANCE 


Recommendation 
To help ensure the prudent use of funds by CHC, the Ministry should: 


° develop measures and benchmarks to monitor and evaluate CHC performance; and 
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° ensure funding agreements include measurable objectives and the results to be achieved 


by CHC for the funding provided. 


Current Status 


According to the Ministry, a preliminary set of measures and performance indicators has 
been developed. Work is underway to identify qualifiers, data definitions, parameters, 
exclusions, and methods of calculation and to test indicators against data from the 
management information system. Specific indicators are to be finalized during 2002. 


The Ministry indicated that 2002/03 funding agreements with CHCs would include a 
preliminary set of indicators to support the CHC Program’s ability to measure whether 
results to be achieved by CHCs are consistent with funded expectations. The January 
2002—October 2002 period will be used to identify and refine benchmarks for CHC 
performance. 


SERVICES AND STAFFING 


Recommendation 
To help ensure that CHCs deliver efficient and economical health care, the Ministry should: 


* obtain and utilize information about the services provided to determine the funding 
level required to provide them; and 


¢ develop guidelines to assist CHCs in determining cost-effective combinations of health- 
care staff. 


Current Status 

The Ministry indicated that difficulties with data extraction have limited the CHC 
Program’s ability to make comparisons among CHCs and analyze trends in service volumes. 
In the meantime, the Ministry has continued to require that CHCs collect the data needed 
for such analysis, such as the number of clients served and services provided and has begun 
generating preliminary reports for analysis. 


According to the Ministry, there are ongoing consultations with stakeholders regarding 
evidence-based practice. For instance, the Ministry is consulting with provider groups and 
other stakeholders to explore combinations of health-care providers that may contribute to 


high-quality and cost-effective health care. 


SERVICES FOR NON-INSURED CLIENTS 


Recommendation 


To enable it to better assess the needs of the CHC Program, the Ministry should obtain complete 
information on the number of non-insured clients served by CHCs and the types of services they 
receive. 
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Current Status 


The Ministry advised us that, until data extraction difficulties are resolved, CHCs will 
submit this information manually to the Ministry. The Ministry indicated that it is working 
on resolving the data extraction difficulties to enable the Ministry to obtain a detailed 
picture of the services received in each CHC by non-insured clients. 


MONITORING SERVICE DELIVERY 


Recommendation 


To help ensure that the services provided by CHCs are of high quality and are provided cost- 
effectively, the Ministry should: 


© conduct regular reviews of CHCs to ensure that expectations are being met; and 


© ensure that CHCs regularly review the quality of care they provide and the services they 


deliver. 


Current Status 


According to the Ministry, the accreditation process for CHCs has been revised to include 
all of the accountability requirements of the Ministry. Agreements with CHCs now require 
CHCs to report their accreditation status each fiscal year. The Ministry also indicated that it 
would reinforce the requirement that CHCs review the quality of care they provide to their 
clients. This requirement is to be included in the new funding agreements with CHCs. 


COMPLAINTS 


Recommendation 

To ensure that complaints concerning CHC are dealt with appropriately, the Ministry should 
require CHCs to have adequate procedures for addressing complaints. 

Current Status 


The revised Policies and Procedures Manual, to be issued in Spring 2002, requires each 


CHC to have a complaints process. Minimum requirements are to be included in the 
manual. 


INFORMATION SYSTEM 


Development and Implementation 


Recommendation 


To help ensure that information systems are properly developed and implemented, the Ministry 
should ensure that appropriate oversight and project management expertise is applied. 
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Current Status 

According to the Ministry, a process has been established to ensure that program 
management and technical expertise are brought to bear on all ongoing projects. In 
addition, the Ministry’s Systems Development Branch (SDB) has the lead role in the 
integrated testing of the next version of the clinical management and data extraction 
software. The Ministry also indicated that CHC program staff were continuing to work 
closely with SDB staff to develop plans to enable cost-effective use of information technology 


inGHGs: 


Access to Information 


Recommendation 


The Ministry should expedite the resolution of any access to information issues to ensure that their 
impact on the new information system is recognized and addressed early in the development 
process. 


Current Status 


The Ministry advised us that a data-sharing agreement between the Ministry and CHCs 
would be finalized in June 2002. 
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MINISTRY OF HEALTH AND LONG-TERM CARE 


4.09—Emergency Health Services 


(Follow-up to VFM Section 3.09, Special Report on 
Accountability and Value for Money—2000) 
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BACKGROUND 


The Ambulance Act governs the provision of ambulance services in Ontario. Under the Act, 
the duties and powers of the Minister of Health and Long-Term Care include ensuring ethe 
existence throughout Ontario of a balanced and integrated system of ambulance services 
and communication services used in dispatching ambulances.” 


Under Local Services Realignment, the responsibility for operating land ambulance services 
was transferred from the province to municipalities, effective January 1, 2001. Since 
realignment, the province funds 50% of the approved costs of land ambulance services, and 
municipalities fund the remaining costs. The Ambulance Act states that every municipality 
will “be responsible for ensuring the proper provision of land ambulance services in the 
municipality in accordance with the needs of persons in the municipality.” The Ministry 
remains responsible for ensuring that minimum standards are met for all aspects of 
ambulance services. During the 2001/02 fiscal year, Emergency Health Services’ 
expenditures totalled approximately $369 million. During the 1999/2000 fiscal year, 
Emergency Health Services’ expenditures, prior to recoveries by the province from 
municipalities for their portion of the operating costs, were approximately $404 million. 


In our 2000 audit, our major concerns were: 


¢ Land ambulance services were being downloaded to municipalities at a time when over 


50% of land ambulance operators were not meeting response time requirements, which 
were based on 1996 actual response times. In addition, these requirements varied 
widely across the province. 


¢ The Ministry estimated that an additional $40 million annually and $11.6 million in 
one-time funding were needed to meet established response time requirements. 


The risk of poor response times was increased because, as stated by the Emergency 
Services Working Group, 36% of the time that hospitals requested redirect 
consideration and critical care bypass, their emergency departments were not at full 
capacity. 

The realigned land ambulance system may not provide a balanced and integrated 
system of services and may be more costly to Ontarians. The Ministry estimated that in 


the year 2000, an additional $53 million would be needed to maintain the existing level 
of service, which was already not meeting response time requirements. 
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We made a number of recommendations for improvement and received commitments 
from the Ministry that it would take corrective action. 


CURRENT STATUS OF RECOMMENDATIONS 


Based on information that we received from the Ministry of Health and Long-Term Care, 
the Ministry has taken some action on all of the recommendations we made in our Special 
Report on Accountability and Value for Money (2000). The current status of each of our 


recommendations is outlined in the following sections. 


LAND AMBULANCE 


Local Services Realignment—Transition 


Recommendation 
The Ministry should ensure that after realignment has been completed, the land ambulance 


program in Ontarto 1s seamless, accessible, accountable, integrated, and responsive. 
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The Ministry should also take corrective action where NeECESSATY. 


Current Status 


In response to a request by the Standing Committee on Public Accounts, which held 
hearings on our 2000 report on this matter, the Ministry completed an assessment report in 
January 2002 on the impact of the Local Services Realignment on the seamlessness, 
accessibility, accountability, integration, and responsiveness of the land ambulance system. 
The Ministry is in the process of addressing the results of the report. Corrective action is to 
be taken where necessary. 


Local Services Realignment—Responsibility for 
Dispatch 


Recommendation 


The Ministry and municipalities should work to ensure that municipal boundaries do not 
impair the delivery of ambulance services to patients or add significantly to costs. 


Current Status 


We were advised by the Ministry that, with the assistance of central ambulance 
communication centres (CACCs), the Ministry has identified initiatives to limit situations 
where ambulances are used to provide services outside their area. Through meetings 
between ministry staff and staff at service providers, these initiatives have been modified to 
ensure that in emergency situations, municipal boundaries will not impact on service 


delivery. 
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The Ministry also advised us that, at the local level, CACC advisory and liaison committees 
continue to meet to address cross-boundary workload pressures and to provide a process for 
managing cross-boundary issues. 


Local Services Realignment—Funding 


Recommendation 


To help ensure that funding provided to municipalities is reasonable and equitable, the Ministry 
should: 


¢ develop a process that assesses relative need and ensures equitable funding across the 
province; and 


¢ define which municipal costs will qualify for provincial funding. 


Current Status 


The Ministry advised us that it has implemented a funding template. This template defines 
the land ambulance costs that the Ministry will share with service providers. All service 
providers have received written notice of their approved cost-sharing grant from the 
Ministry and have been advised that, subject to the availability of funds, adjustments to their 
approved cost-sharing grant may be requested through a business case. Each business case is 
to be assessed by ministry staff. 


Response Times—Ambulance Response Times 


Recommendation 


To help ensure that ambulance response times for emergencies meet the needs of patients 
throughout the province, the Ministry, together with the municipalities, should: 


° review current response time requirements for reasonableness and consistency and, 


where necessary, make adjustments; and 


take appropriate corrective action where specified response time requirements are not 
met. 


Current Status 


The Ministry advised us that it reached an agreement with the Association of Municipalities 
of Ontario and the Land Ambulance Implementation Steering Committee on a response 
time improvement framework. The framework was initially tested in six municipalities, and 


the results analyzed. On August 17, 2001, a revised framework was sent to all service 
providers. 


All response time improvement framework submissions from service providers were received 
by the Ministry and analyzed. Requests for additional provincial resources for response time 
improvements were submitted to the Ministry. These were being reviewed by ministry staff. 
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The Ministry also indicated that it has conducted a comprehensive review of land 
ambulance response times and is now providing municipalities with access to response time 


statistical data. 


Response Times—Dispatch Response Times 


Recommendation 
To better meet the needs of patients, the Ministry should: 


° establish dispatch response time standards; 
© monitor whether these standards are being met; and 


° take timely corrective action where necessary, 


Current Status 


The Ministry advised us that performance agreements were in place with five of the eight 
central ambulance communication centres (CACCs) not operated by the Ministry. These 
agreements include dispatch standards. Agreements with the remaining three were being 
pursued. Discussions were also underway to add standards for dispatch response times to the 
performance contracts of the managers of ministry-operated CACCs. 
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The Ministry advised us that the staffing and technological enhancements needed for 
CACCs to meet minimum dispatch response time standards have been requested via the 
Ministry's 2002/03 Business Planning Allocation process. 


The Ministry also indicated that its field staff monitor the performance of CACCs by 
reviewing incident reports relating to delays in dispatching and service delivery and that an 
analysis of the 2001 CACC reaction time standards is planned for spring 2002. 


Response Times—Redirect Consideration and Critical 
Care Bypass 


Recommendation 


The Ministry should analyze the impact of redirect consideration and critical care bypass on 
ambulance services, including response times for subsequent patients, and, where necessary, take 
appropriate corrective action. 


Current Status 


In October 2001, the Ministry implemented the Patient Priority System (PPS) to replace 
redirect consideration and critical care bypass. Under PPS, a patient’s degree of distress is 
categorized by paramedics in accordance with the Canadian Triage Acuity Scale (CTAS), 
thereby ensuring that paramedics, ambulance dispatch personnel, and hospital emergency 
personnel use the same language to describe the needs of emergency patients. In addition, 
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while hospitals can communicate their degree of emergency department pressure to 
ambulance dispatch centres using categories defined by CTAS, PPS requires that critical 


ambulance patients are to always be taken to the closest emergency department. 


Dispatch Priority 


Recommendation 


The Ministry should ensure that central ambulance communication centres appropriately assess 
and prioritize patient needs. 


Current Status 


The Ministry has hired 11 training officers who have commenced work in training dispatch 
staff in the assessment and prioritization of patient needs. 


The Ministry indicated that it has discontinued the project to enhance the Dispatch Priority 
Card Index (DPCI) protocol in the current computer-aided dispatch environment because 
of concerns raised about amending the current system on the existing computer platform. 
The Ministry plans to restart this project once the replacement computer-aided dispatch is 
implemented (late 2002/early 2003). 


Performance Monitoring 


Recommendation 


To help ensure that the land ambulance system effectively meets patient needs, the Ministry 
should: 


° research systems to analyze operator performance, including its impact on patient 
outcomes; and 


© take corrective action where necessary. 


Current Status 


The Ministry has developed and implemented a certification process that focuses primarily 
on patient care provided by service providers. The certification process provides for action 
where a contravention of standards has occurred. According to the Ministry, follow-up visits 
are being conducted on applicants where a breach of certification criteria is found. Failure 
on the part of an existing operator to meet criteria may lead to a revocation of certificate. 


Performance Monitoring—Service Reviews 


AMBULANCE OPERATOR SERVICE REVIEWS 


Recommendation 


To help ensure that ambulance operators meet ministry requirements, the Ministry should: 





376 2002 Annual Report of the Provincial Auditor of Ontario 


—————————————————————— 


* consider performing certification reviews without advance notice to increase assurance 
of consistent quality of practice by operators; 


° have a co-ordinated follow-up of all deficiencies identified during certification reviews 
on a timely basis; and 


° clarify the circumstances when a formal investigation of an operator is required and 
when a certificate should be revoked. 


Current Status 


The Ministry indicated that, in addition to instituting an inspection process with notice 
given, it is now conducting visits to follow up on certification reviews with little or no notice 
given depending on the nature of the breach of certification criteria. A standard 
implementation and follow-up process addressing the deficiencies identified during 
certification reviews is being implemented by the Emergency Health Service Branch’s field 
managers. The Ministry will consider revoking a certificate where a contravention of 
Ambulance Act standards has occurred. 


CENTRAL AMBULANCE COMMUNICATION CENTRE AND BASE 
HOSPITAL REVIEWS 


Recommendation 

To help ensure that emergency patients’ needs are being effectively and consistently addressed, the 
Ministry should: 

° review central ambulance communication centres and base hospitals within reasonable 


timeframes; and 


° resolve all identified deficiencies on a timely basis. 


Current Status 
According to the Ministry, reviews have now been conducted on each base hospital and will 


continue to be conducted on a three-year cycle. 


The Ministry advised us that deficiencies found during reviews at base hospitals are 
discussed with base hospital staff, corrective action plans are developed jointly, and the 
Ministry monitors whether corrective action is taken by the base hospital. A tool to review 
central ambulance communication centres is being developed and should be approved for 
use in 2002. 


Performance Monitoring—Complaints 


Recommendation 
To better enable it to assess whether complaints are satisfactorily resolved, the Ministry should: 
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© establish clear lines of responsibility for following up on deficiencies identified in inves- 
tigation reports; and 


¢ ensure that follow-ups are completed and documented. 


Current Status 


According to the Ministry, a draft process for investigating complaints about ambulance 
service providers has been referred for comments to the Freedom of Information 
Subcommittee of the Land Ambulance Implementation Steering Committee. There is also 
ongoing dialogue on the matter between the Ministry and service providers. 


The Ministry indicated that it has forwarded a funding request to Management Board of 
Cabinet for additional staff to ensure that follow-ups are completed and documented. 


Non-Emergency Inter-Institutional Transfers 


Recommendation 


The Ministry and municipalities should jointly develop and ensure standards are in place that 
address passenger safety and encourage the use of the most cost-effective resources for transferring 
non-emergency patients. 


Current Status 


According to the Ministry, the Land Ambulance Implementation Steering Committee 
identified inter-institutional transfers as one of the highest priorities on its agenda. The 
Ministry has hired a consultant to undertake a wide-ranging independent study of how 
inter-institutional transfers should be managed, funded, and provided. The consultant is 
expected to provide a report by late spring 2002. 


AIR AMBULANCE PROGRAM 


Use and Selection of Aircraft 


Recommendation 


The Ministry needs to better demonstrate through proper documentation that air ambulances 


are used appropriately and that the aircraft selected meet patient needs in the most economical 
manner. 


Current Status 


The Ministry informed us that it has installed a new system that considers level of care and 
patient compatibility using well-documented information on the patient’s medical condition 


and flight requirements. This will help ensure that air ambulances are used appropriately 
and economically. 
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Response Times 


Recommendation 


1o help ensure that air ambulance dispatch and response times meet the needs of patients, the 
Ministry should: 


* develop, track, and monitor air ambulance dispatch response time standards; 


° track and monitor contracted air carrier response times and take corrective action 
when necessary; and 


* ensure air carrier contracts contain appropriate penalties for not meeting required 
response times. 


Current Status 


In 2001, the Ministry developed response time standards for air ambulance dispatch. 
Reports generated by the existing dispatch computer system showed that dispatch response 
time standards were being met in 2001. The Ministry indicated that it is likely a year away 
from integrating its new Dispatch Flight Assist Call Tracking System with flight planning. In 


the meantime, the Ministry was continuing to manually monitor dispatch response times. 


| Follow-up Section 4.09 | 





According to the Ministry, the project officer responsible for the management of contracts 
with carriers now monitors and verifies invoices and incident reports, including cases where 
carriers have not met required response times. The project officer is also responsible for 
imposing related penalties on operators in accordance with their contracts. 


Inspecting Service Providers 


Recommendation 


1o help ensure that the air ambulance program is providing safe and quality services at an 
appropriate cost, the Ministry should: 


* conduct inspections and evaluations of air ambulance providers in accordance with 
ministry policies and procedures; 


* track and analyze air ambulance use and performance data; and 


° take corrective action when Necessary. 


Current Status 


The Ministry indicated that it has increased the number of inspectors and is providing 
introductory training to all new inspectors. The responsible project officer works with 
appropriate ministry staff to set standards and reviews the re-certification of air operators. 
Ministry inspectors expect to be increasing inspection activities in spring 2002. 
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The Ministry advised us that it is likely a year away from implementing a new integrated 
system that will permit it to report on air ambulance use by type of aircraft and type of calls 
dispatched. 


According to the Ministry, the performance of service providers continues to be monitored, 
and no areas of non-compliance have been identified to date. 


Patient Billings 


Recommendation 


To help ensure that, where applicable, all patients are billed equitably and outstanding amounts 
are collected, the Ministry should establish effective procedures to: 


© ensure that all patients who should be billed are identified and invoiced in a timely 
manner for the total cost of the service provided, regardless of the air carrier used; and 


° collect outstanding accounts on a timely basis. 


Current Status 


According to the Ministry, where applicable, patients are now being charged for the total 
costs of services provided. Billings are sent to patients or their insurance companies within 
30 days of the Ministry receiving the related invoice/information from hospitals. Second 
and third reminders are sent at 30-day intervals. For patient and insurance company 
invoicing, there is a cost-recovery system in place whereby each flight is reviewed and the 
amount to be billed is determined. 


For the collection of outstanding accounts, an agreement with the Collection Management 
Unit of Management Board Secretariat is being finalized. 
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MINISTRY OF HEALTH AND LONG-TERM CARE 


4.10—Health Service Organization and 
Primary Care Network Programs 


(Follow-up to VFM Section 3.10, Special Report on 
Accountability and Value for Money—2000) 


— 
BACKGROUND 


The Health Service Organization (HSO) Program was established in 1973, and the Primary 
Care Network (PCN) Program was established in 1999. Each HSO and PCN comprises 
physicians who have agreed to provide a defined set of primary health-care services to their 
enrolled patients. During the 2001/02 fiscal year, the Ministry provided transfer payments 
totalling approximately $45 million to HSOs and $36 million to PCNs. 


Our major concerns with the programs were: 
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* HSO patient rosters had only been verified once, despite the fact that approximately 
8,000 of the 18,000 patients verified at that time proved to be ineligible and were 
removed from HSO rosters. 


¢ The Ministry had not assessed whether it was receiving value for money for the more 
than $20 million in annual funding it provided to the Group Health Association. 


* Expansion of the PCN Program, to include 80% of eligible family doctors, was being 
planned while evaluations of the pilot PCNs were still not completed. 


* Capitation (per person) funding rates did not take into account factors that may affect 
the need for primary health care, such as patients’ medical histories. 


We made a number of recommendations for improvement and received commitments 
from the Ministry that it would take corrective action. 


CURRENT STATUS OF RECOMMENDATIONS 


According to information obtained from the Ministry, the Ministry has taken some action 
on all of the recommendations made in our Special Report on Accountability and Value for 
Money (2000) The current status of each of our recommendations is as follows. 
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HEALTH SERVICE ORGANIZATIONS 


Roster Verification and Negation 


Recommendation 


To help ensure that payments to HSOs are made only for patients actually receiving their 
primary care from the HSO, the Ministry should: 


© implement regular verification procedures for HSO rosters; and 


© where practical, apply negation for primary care services provided to rostered patients 
by alternatively funded physicians. 


Current Status 


According to the Ministry, a regular roster verification process has been implemented. This 
process is to be compatible with the new PCN system. In 2001, the Ministry sent each 
HSO a list identifying the patients to be verified and the letters that were to be sent to these 
roster members, who represented 5% of each HSO’s roster. All HSOs have submitted their 
results to the Ministry, and roster changes were in the process of being implemented. 


The Ministry also advised us that it had performed audits at 58 HSOs in 2000 and 53 
HSOs in 2001 to ensure that patient signatures were on enrolment forms. Signature audits 
for 2002 have been completed at 20 HSOs, and the remainder were to be completed by 
October 2002. We understand that these audits are being followed up on to ensure full 
compliance with ministry requirements. 


The Ministry indicated that, where feasible, negations (deductions from subsequent 
payments) would be implemented for primary care services provided to HSO-rostered 
patients by alternatively funded physicians. 


Roster and Billing Limits 


Recommendation 


To ensure that HSOs are paid in accordance with agreed-upon roster sizes and fee-for-service 
billing limits, the Ministry should monitor HSO compliance with those limits. 


Current Status 


According to the Ministry, fee-for-service and roster limit recovery notices for the 1997/98 
and 1998/99 fiscal years were sent to HSOs in October 2000. This resulted in recoveries 
totalling approximately $111,000. The Ministry advised us that any outstanding amounts 
are being collected in accordance with accounts receivable guidelines. 


The Ministry also indicated that the amounts to be recovered for the 1999/2000 and 
2000/01 fiscal years were to be calculated in the summer of 2002. 
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Institutional Substitution Program Grants 


Recommendation 
To help ensure that value for money is received from Institutional Substitution Program grants, 
the Ministry should: 


* examine the appropriateness of the grants; and 


° review the grants being provided and, where necessary, make appropriate adjustments. 


Current Status 

According to the Ministry, Institutional Substitution Program (ISP) grants were reviewed 
starting in July 2001. However, because of the lack of historical information, 
recommendations were made to continue the program and review the grants again after the 
2001/02 activity reports were received in June 2002. 


The Ministry also advised us that new templates for preparing activity reports were sent to 
grant recipients in March 2002. As part of the renegotiation of the ISP agreements, the 
grants will be reviewed with a view to reforming the grants to align with standardized targets 
and programs. 


Performance Measurement and Reporting 


Recommendation 


The Ministry should complete its study comparing health-care utilization and costs in health 
service organizations with fee-for-service practices and ensure that the results are considered in 


the implementation of the PCN Program. 


Current Status 


The Ministry advised us that ministry staff are continuing to work with the Centre for 
Health Economics and Policy Analysis at McMaster University to provide historical roster 
information that is needed to complete the study comparing health-care utilization and costs 
in health-service organizations with fee-for-service practices. 


Group Health Association 


Recommendation 
To help ensure that it is receiving value for the funding it provides to the Group Health 
Association (Association), the Ministry should: 


* require the Association to provide it with sufficient information so the Ministry can 
assess whether the Association’s programs and services were delivered efficiently and 
effectively; 


° ensure that the accuracy of the Association’ roster is verified periodically; and 
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e assess the reasonableness of the outside use of medical services by the Association 5 rostered 
patients and implement a process that enables the Ministry to reduce payments to the 
Association where warranted, as is currently done in the Health Service Organization 


Program. 


Current Status 


According to the Ministry, the Ministry is negotiating a new contract with the Association to 
address the issues we reported in 2000, and these negotiations are to be finalized by 
September 30, 2002. The new agreement is to include mechanisms for verifying the roster. 


The Ministry also advised us of the following: 


¢ The Ministry has requested a review of certain aspects of the Association’s current 
operations to better understand how the Association operates and to address deficiencies 
as measured against government accountability guidelines. Details of the Association's 
program management, funding sources, and partnerships are also being requested to 
ensure effective service delivery. 


¢ Established ministry programs are to be used as models for setting acceptable 
benchmarks and for initiating procedures to ensure the accuracy of the Association’s 
roster and to assess and control the use of outside medical services by the Association’s 
patients. 


PRIMARY CARE NETWORK PROGRAM 


Implementing Primary Care Networks 


Recommendation 


To help ensure that the PCN Program provides improved primary health care in a cost-effective 
manner, the Ministry should, in line with the planned expansion of the Program, carefully assess: 


° the interim reports on the evaluation of the PCN pilots; 


° relevant experiences of other jurisdictions; and 


the results of the ministry-funded study comparing health-care utilization and costs in 
health service organizations and fee-for-service practices. 


Current Status 


Primary care networks are now known as family health networks, which are supported by 

the Ontario Family Health Network (OFHN). The OFHN was created by the province in 
March 2001, and its role specifically includes providing family physicians with information, 
administrative support, and technology funding to support the voluntary creation of family 


health networks in their communities, According to the Ministry, the OFHN is the agency 
that will lead the expansion of family health networks. 
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The Ministry has received two formal reports on the evaluation of primary care reform 
pilots in Ontario. The evaluation reports were shared with the OFHN. The Phase report 
assesses the implementation process and presents observations and findings to be considered 
in developing an implementation plan to encourage and support the expansion of family 
health networks. The Ministry indicated that this report will assist the OFHN in adjusting 
its ongoing support to the existing pilot networks if possible and assist physicians in the 
continuing operation of their networks. The Phase 2 report describes and assesses the 
progress that is occurring, the barriers to progress, and the opportunities for improvement. 
It examines results and presents recommendations in line with reform objectives. Its findings 
have been considered in implementing the new family health networks and in the 
continuing administration of the pilot networks. 


Phase 3 is intended to evaluate reform results or outcomes: its main focus is to be the impact 
of primary care reforms in the pilot communities and the outcomes achieved by the pilots. 
The Phase 3 report is to provide a summary of key findings from all three phases of the 
evaluation, an assessment of goal achievement, and recommendations and is to bring 
together all of the evaluation’s findings in order to formulate conclusions. We were advised 
that the OFHN and the Ministry have reviewed the draft Phase 3 report and have 
provided corrections regarding factual errors. The Ministry anticipates that the “final draft 
report will be released in the very near future.” 


The Ministry is continuing to participate on federal, provincial, and territorial committees 
that are studying primary care reform initiatives in other jurisdictions. 


As mentioned earlier, ministry staff are working with the Centre for Health Economics and 
Policy Analysis to provide historical roster information that is needed to complete the study 
comparing health-care utilization and costs in home service organizations and fee-for-service 
practices. 


Capitation Rates 


Recommendation 


10 help ensure that funding of primary care networks is equitable, the Ministry should consider 
options, including those utilized by other jurisdictions, for adjusting capitation rates to equitably 
reflect the level of services being provided. 


Current Status 


According to the Ministry, the Centre for Health Economics and Policy Analysis (CHEPA) 
at McMaster University has completed a draft preliminary literature review of policy options 
for physician remuneration. CHEPA’s paper includes options for adjusting capitation rates 
and a survey of how capitation rate modification has been addressed in other jurisdictions. 
We were advised that ministry staff have reviewed the paper and will be working with 
CHEPA to explore the potential use of a model developed by Johns Hopkins University that 
was pilot tested in British Columbia. 
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The Ministry advised us that its ongoing negotiations with the Ontario Medical Association 
have included an examination of the appropriateness of current capitation rates and means 
for the diversification of provider remuneration. This resulted in the development of two 
compensation models. 


Referrals to Specialists 


Recommendation 


To help ensure that the process for referring patients to specialists is cost effective, the Ministry 
should: 


© determine whether different methods of funding primary health care affect referrals to 


specialists; and 


¢ develop methods for monitoring and improving the quality of the referral process. 


Current Status 
We were advised that the University of Toronto’s Hospital Management Unit has been 


working with the Ministry to provide advice and recommendations on appropriate 
performance measures for PCNs that may include tracking of referral rates to specialists. 
Further details on this initiative are provided in the Performance Measurement and 
Reporting section of this follow-up report. 


The Ministry, through the Ontario Health Insurance Plan (OHIP) system, has performed 
some preliminary analysis of rates and patterns of referrals by physicians to specialists with a 
view to a more extensive examination. The Ministry indicated that it is establishing linkages 
between the University of Toronto and the Ministry’s generic alternate payment programs 
projects to ensure that it will have a comprehensive and improved system to monitor 
performance, enhance accountability, and manage costs. 


The Ministry also indicated that the evaluation of primary care reform pilots includes 
provider surveys that are intended to furnish qualitative information on referral patterns. 


Interdisciplinary Primary Health Care 


Recommendation 
To help ensure that PCNs deliver cost-effective primary health care, the Ministry should: 


° thoroughly evaluate the experiences of other jurisdictions to identify best practices that 
warrant consideration in Ontario; and 


* provide guidance on the combinations of health-care providers that are required to 


provide high-quality and cost-effective health care. 
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Current Status 


According to the Ministry, it continues to participate on federal, provincial, and territorial 
committees that are studying primary care reform initiatives, including interdisciplinary care 
models in other jurisdictions. The goal of this continued participation is to develop an 
appropriate strategy for Ontario. 


We were advised that ministry staff and the Chair of the OFHN have met with a large 
number of groups representing health workers to obtain their input on collaborative health 
care. In addition, the Ontario College of Family Physicians has submitted a proposal to the 
OFHN regarding collaborative working relationships and the roles of nurse practitioners 
and other primary care disciplines that identifies factors that contribute to successful 
collaboration and makes recommendations regarding liabilities and funding issues. The 
Ministry advised us that it has applied to the federal government for funding from the 
Primary Health Care Transition Fund and that it is intending to more than double the 
number of nurse practitioners working in Ontario. The Ministry anticipates that many of 
these positions will be placed in family health networks. 


Roster Limits and Verification 


Recommendation 


To help ensure that roster limits are reasonable, the Ministry should research best practices in 
other jurisdictions and establish a sound basis for setting PCN roster limits, 


The Ministry should also ensure that an effective verification process 1s implemented for PCN 


patient rosters. 


Current Status 


According to the Ministry, roster limits in the current primary care pilots as well as 
experiences in other jurisdictions are to be evaluated to determine how to set appropriate 
roster limits. The Ministry’s current agreement with the Ontario Medical Association states 
that family health network template agreements will have no roster limits if a physician 
personally and directly provides the majority of primary care medical services to the patients 
rostered to that physician. 


Annual audits to verify each family health network’s physician’s roster of enrolled patients are 
to be conducted either by the Ministry or the Ontario Family Health Network. Family 
health network agreements also contain a provision for enrolment verification beyond 
annual audits. Roster verification is to include letters to patients, the listing of patients who 
have been selected for enrolment verification, and random audits to ensure that patient 
letters are kept on file by the physician. The Ministry indicated that changes to the OHIP 
system are to be made to support the enrolment verification process. The Ministry expects 
that the first enrolment verification cycle will be completed by the fall of 2002, at which 
time the cycle will be repeated. 
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Performance Measurement and Reporting 


Recommendation 


To enable it to assess the quality of care provided by PCNs, the Ministry, in collaboration with 
primary health-care providers, should develop: 


© appropriate performance measures and standards; and 


¢ a health performance information system that meets the needs of the Ministry and 
PCNS. 


Current Status 


As mentioned earlier, we were advised that the Ministry has been working with the 
University of Toronto to develop appropriate performance measures for PCNs. These may 
include tracking of referral rates to specialists, waiting times, and services provided by nurse 
practitioners, as well as other measures. This research included input from experts. In June 
2001, a report was presented to the Ministry indicating outcomes, identifying performance 
issues, and making recommendations. 


Over the next two years, the University is to field-test performance indicators by identifying 
barriers to data collection and suggesting means to overcome the barriers. The testing is also 
intended to provide an opportunity to develop performance measures from the indicators 
identified, identify data sources, and investigate possible standards for the indicators. A 
seminar for primary care physicians on these performance indicators is to be developed. 


The Ministry indicated that it has begun work on the Generic Alternate Payment Programs 
project to develop systems and business practices to enhance development, management, 
enforcement, payment, and accountability. The project team has begun its work to review 
alternative payment contracts and data-capture systems with the intent of developing and 
implementing processes for standardized alternative payment contracts. Linkages are also to 
be established to ensure that the Ministry will have a comprehensive and improved system 
to monitor performance, enhance accountability, and manage costs. 
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MINISTRY OF HEALTH AND LONG-TERM CARE 


4.11—Ontario Midwifery Program 


(Follow-up to VFM Section 3.11, Special Report on 
Accountability and Value for Money—2000) 
enn 


BACKGROUND 


On December 31, 1993, midwifery became a regulated health profession in Ontario. The 
Midwifery Act defines the practice of midwifery as the assessment and monitoring of women 
during pregnancy, labour, and the post-partum period and the provision of care to women 
and their babies during normal pregnancy, labour, and post-partum period. 


The Ontario Midwifery Program was established in 1994 to fund professional midwifery 
services. Based on information provided by midwifery practice groups, the Ministry 
estimated that midwives attended approximately 6,000 births in the 2000/01 fiscal yeat. 
The Ministry expects that, by the 2003/04 fiscal year, midwives will be attending about 
12,000 births annually in the province. 


For the 2001/02 fiscal year, the Ministry provided approximately $28.6 million to fund the 
provision of midwifery services. In 1999/2000, it provided $17 million. 


In our Special Report on Accountability and Value for Money (2000), our major concerns 
with the Midwifery Program were: 


° There was a lack of adequate information to determine whether the objectives of the 
Program were being met. 


* The Ministry had not assessed the cost-effectiveness of the delivery and funding model 


for midwifery services. 


° The process for referring midwifery clients to specialists may have been creating 
additional costs for the health-care system. 


We made a number of recommendations for improvement and received commitments 
from the Ministry that it would take corrective action. 


CURRENT STATUS OF RECOMMENDATIONS 


According to information obtained from the Ministry, the Ministry has taken some action 
on all of the recommendations we made in our Special Report on Accountability and Value 
for Money (2000). The current status of each of our recommendations is as follows. 
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ASSESSING COST EFFECTIVENESS 


Recommendation 

To help assess the quality of midwifery services and assess whether these services are delivered 
efficiently and effectively, the Ministry, along with the College of Midwives of Ontarto and the 
Association of Ontario Midwives, should: 


© determine what information is needed to make these assessments; and 


° ensure that the information 1s collected and analyzed. 


Current Status 


We were advised by the Ministry that in June 2002 it completed a feasibility study on a 
midwifery management information system, which it worked on with the Association of 
Ontario Midwives, the College of Midwives of Ontario, and the Midwifery Education 
Program. The Ministry indicated that it will implement the study's recommendations in 
phases beginning in the 2002/03 fiscal year. 


EQUITABLE ACCESS 


Recommendation 


To determine whether the Midwifery Program is meeting its objective of ensuring greater equity 
of access to midwifery services, the Ministry should: 


¢ clearly define the meaning of “greater equity of access’; 


¢ assess the impact of the allocation of midwifery services in the agreements between local 
agencies and midwifery practice groups; 


° review the arrangements that permit midwives to have private clients; and 


¢ clearly define “resident” for the purposes of eligibility for publicly funded midwifery 


SEYVICES. 


Current Status 


We were advised that, as a first step to measuring access to midwifery services, the Ministry 
requires that local agencies report to it the source of the referral for clients obtaining 
midwifery care. This enables the Ministry to track the means of access to midwifery services. 
In addition, to monitor access to midwifery services, the Ministry also requires that local 


agencies report to it the number of clients being cared for by ministry-funded midwifery 
practice groups. 


The Ministry indicated that it continues to monitor client access to midwifery services. Data 


continue to show that current agency referral patterns and the arrangements allowing 
midwives to have private clients are not interfering with the general population’s access to 
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publicly funded midwifery services. However, demand for midwifery services continues to 
exceed the supply of midwives. 


The Ministry advised us that it will continue to monitor access by reviewing data supplied 
by agencies on a quarterly basis. 


The Ministry indicated that although the midwifery definition of resident is not identical to 
that of the Health Insurance Act, upon review, the Midwifery Program definition provides 
the most appropriate level of access to midwifery services for the population being served. 


REFERRALS TO SPECIALISTS 


Recommendation 


The Ministry should ensure that the current process for referring clients of midwives to specialists 
does not result in unnecessary visits to family physicians or hospital emergency departments. 


Current Status 


The Ministry, in collaboration with the Ontario Medical Association and the Association of 
Ontario Midwives, established, effective April 1, 2002, new fee codes under the Ontario 
Health Insurance Plan’s Schedule of Benefits that permit the same compensation to 
specialist physicians for midwife requests as for requests for consultation by physicians. 
According to the Ministry, this will alleviate the barrier that prevented midwifery clients 
from directly accessing specialists. 
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MANAGEMENT BOARD SECRETARIAT 


4.12—Movable Assets 


(Follow-up to VFM Section 3.12, Special Report on 
Accountability and Value for Money—2000) 





BACKGROUND 


Ministries’ movable assets consist primarily of office furniture and equipment, such as 
photocopiers and fax machines; information technology (IT) equipment, including desktop 
and notebook computers, printers and other peripherals; audio-visual equipment, such as 
televisions, VCRs, and cameras; and motor vehicles. 


During the 1998/99 fiscal year—the last year for which this information was available—all 
ministries spent approximately $500 million on movable assets, much of which was spent on 
IT equipment in preparation for Y2K. However, the total value, type, and quantity of 
movable assets on hand were not known because ministries did not keep adequate records 
in that regard. 


In 2000, we conducted our audit work at five ministries and issued a detailed report to each 
deputy minister of the ministries included in our scope. In addition, since Management 
Board Secretariat (MBS) develops government-wide policies and standards for acquiring 
and managing movable assets and had entered into a number of government-wide standing 
agreements for IT equipment acquisitions, we summarized in Section 3.12 of the Special 
Report on Accountability and Value for Money (2000) the more significant issues addressed in 
the individual ministry reports. 


We concluded that although ministries generally followed the process recommended by 
MBS for acquiring IT equipment and used the MBS standing agreements with various 
manufacturers for their equipment acquisitions, doing so did not ensure that they received 
value for money spent because: 


communication between MBS and the ministries regarding the pricing provisions 


under the MBS standing agreements was insufficient to ensure that ministries were 
obtaining the best prices; 


there was no requirement for the ministries to demonstrate that the makes and models 
of equipment acquired were the most appropriate and cost effective for their needs; and 


the cost effectiveness of leasing as the preferred option for most of the equipment 
acquired was not established, and the terms of the standing lease agreement were not 


competitively entered into nor was value for money achieved as a result of prepaying or 
overpaying lessors. 


We also concluded in 2000 that movable assets were not adequately managed because: 
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* accurate and up-to-date listings of all owned and leased movable assets were not 
maintained; 


* the existence and efficient deployment of movable assets were not periodically verified; 
and 


* missing or underutilized assets were not followed up ona timely basis. 


We made a number of recommendations for improvement and received commitments 
from MBS and the ministries that they would take corrective action. 


CURRENT STATUS AND RECOMMENDATIONS 


MBS had taken or was in the process of taking corrective action on our two published 
recommendations. Their status as of June 30, 2002 is as detailed in the following sections. 


ACQUISITION OF INFORMATION TECHNOLOGY 
EQUIPMENT 


Information Technology Equipment Pricing 


Recommendation 


10 ensure that ministries acquire IT equipment in compliance with Management Board of 
Cabinet directives, Management Board Secretariat (MBS) should work with ministries to 
ensure competitive prices are achieved until a new competitive pricing structure ts established 


through implementation of the proposed Total Cost of Ownership concept. 


Current Status 
We were advised that MBS received Management Board of Cabinet approval in June 2002 


for its government-wide desktop equipment and services strategy. A request for proposals to 
establish the corporate desktop management services and products vendor-of-record 
arrangement is scheduled to be issued in August 2002. The use of this vendor-of-record 
(VOR) arrangement will be mandatory for all Information and Information ‘Technology 
clusters and ministries, with the exception of designated “optional services”, which may also 
be provided by internal Ontario Public Service staff. Examples of such optional services 
include: installation and de-installation of desktops; moves, adds, and changes; maintenance 
and break-fix services; virus eradication; and software control and distribution. 


The VOR agreement is designed to ensure that pricing is continuously in line with market 
conditions to provide best value for money spent. Such pricing 1s to be validated through 
regular benchmarking. This initiative is intended to lead to lower support and application 
costs and an overall lower total cost of ownership. 
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MBS also established an interim policy in November 2000 that provides direction 

regarding the use of existing VOR agreements and that, in particular, requires buyers to 
aggregate their demand in order to achieve volume discounts through a formal second-stage 
competitive process. 


Information Technology Equipment Leasing 


Recommendation 


To help ensure that IT acquisitions are financed in an economical manner, as required by 


Management Board of Cabinet directive, MBS should: 


© formally assess the desirability and cost effectiveness of leasing IT equipment in compari- 
son to other forms of financing or purchasing; and 


© follow a competitive process when renewing its standing agreement for lease-financing 
Services. 


Current Status 


In the fall of 2001, MBS financial analysts compared the total cost of ownership of desktop 
assets under both a lease and purchase scenario using data from two IT clusters. This 
comparison revealed that, while there was not a material difference between the lease and 
purchase scenarios over the standard three-year asset life cycle, the leasing scenario did 


provide a slight cost advantage. In addition, MBS noted a number of other advantages 
associated with leasing. 


With respect to the second part of the recommendation, MBS advised us that it had 
completed a competitive procurement process to establish a new corporate agreement for 
lease-financing services for IT equipment. The agreement came into effect on 

February 1, 2002 and is mandatory for use by all IT clusters and all ministries. 


In the agreement, five vendors have been selected to provide financing services to ensure 
that there is continuous competition among those vendors to provide optimal lease rates. All 
five vendors have agreed to identical terms and conditions in order to ensure consistency in 


leasing practices and asset management. All existing contracts are in the process of being 
replaced by this agreement. 
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MINISTRY OF NATURAL RESOURCES 


4.13—Forest Management Program 


(Follow-up to VFM Section 3.13, Special Report on 


Accountability and Value for Money—2000) 
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BACKGROUND 


Under the Crown Forest Sustainability Act 1994, the Ministry is responsible for ensuring the 
long-term health of Ontario’s Crown forests. However, under the Act, licensed forest 
management companies became directly responsible for forestry planning, harvesting, and 
renewal. The legislation also includes sanctions and penalties for non-compliance by forest 
management companies. The Ministry's role in ensuring the long-term health of Crown 
forests is progressively becoming one of overseeing the activities of forest management 
companies. 


In the 2001/02 fiscal year, the Ministry spent $58.7 million on forest management. In 
1999/2000, it spent $70.8 million. 


In 1999/2000, the Ministry had collected stumpage charges totalling $155.7 million. In 
addition, two trusts set up to reimburse forestry management companies for renewal 
expenditures paid out an additional $104.8 million for forest renewal and related activities. 


In our Special Report on Accountability and Value for Money (2000), we concluded that the 
Ministry did not have sufficient information to adequately meet its obligation to annually 
report on the management of Ontario’s Crown forests. In addition, the Ministry had not yet 
completed its transition from directly managing many aspects of forestry to implementing 
appropriate oversight and monitoring procedures to ensure that forestry companies comply 
with legislation and ministry policy and that the long-term health of Ontario’s Crown forests 
is managed with due regard for economy and efficiency. Specifically, at that time we noted: 


* The Ministry had not reported annually on the management of Ontario’s Crown forests 
as required by the Environmental Assessment Board. In addition, sufficient information 
was unavailable in some forest management units to properly assess the harvest area 
successfully renewed, which is a key measure of forest sustainability. 


* Over half the district offices reported that forest management companies had 
significantly over- or under-harvested. The actual harvests over the past six years for each 
of the 68 management units had ranged from 20% to 122% of planned levels. 


* In areas where the Ministry had continued to perform compliance inspections after the 
responsibility for such inspections was delegated to the forest management companies, 
munistry inspectors were finding significantly more violations than industry inspectors. 
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© Ministry district offices had been inconsistent in imposing penalties for non-compliance 
and stricter penalties were likely warranted where warnings and less severe measures 
were not having the desired deterrent effect. 


Accordingly, we made a number of recommendations to improve the Ministry's procedures 
for ensuring the long-term health of Ontario’s Crown forests and received commitments 
from the Ministry that it would take corrective action. 


CURRENT STATUS OF RECOMMENDATIONS 


Based on information received from the Ministry, some action has been taken on all of the 
recommendations made in our Special Report (2000). The current status of each of our 
recommendations is outlined below. 


MEASURING AND REPORTING ON FOREST 
SUSTAINABILITY 


Reporting Requirements 


Recommendation 


The Ministry should report annually on the management of Crown forests, as required under the 
Environmental Assessment Act, so that any necessary corrective action can be taken on a 
timely basis. 


Current Status 

The Ministry informed us that annual reports on forest management have been released for 
the years 1996/97 through to 1999/2000 and that they intend to continue producing this 
report annually and to release it within the eighteen months following the end of the fiscal 
year. In addition the 2001 State of the Forest Report (covering 1995-2000 ) was tabled in 


the Legislature and released to the public in 2002 (state of the forest reports are required 
every five years). 


Measures Of Effectiveness 


Recommendation 


To ensure that sound conclusions can be drawn on the state of renewal of Ontario s Crown forests, 
the Ministry should require that the necessary forest assessments be carried out in each forest 


management unit. 
Current Status 


The Ministry informed us that a number of procedures have been put into place to assess 
key forest performance measures, such as the harvest area successfully renewed. For instance, 
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the Forest Information Manual was approved by regulation under the Grown Forest 
Sustainability Act on April 25, 2001; and this manual outlines the requirements for forest 
renewal assessment and reporting. In addition, the Ministry released the Silvicultural 
Effectiveness Monitoring Manual for Ontario in November 2001. This manual describes the 
requirements and expectations for forest regeneration objectives, renewal survey 
methodologies, and the reporting of silviculture effectiveness. 


The Ministry will continue to rely on the comprehensive, independent audits that are 
performed on each forest management unit every five years. These audits provide an 
assessment of the success of forest renewal and of the health of the management unit’s 
forests. 


Finally, the Ministry informed us that in 2001, independent audits were conducted in 20 of 
the 68 forest management units, and all 20 units reported on the harvest area successfully 
renewed. 


FOREST MANAGEMENT OPERATIONS 


Forestry Information 


Recommendation 


To comply with legislation and the 1994 requirements of the Environmental Assessment Board 
and to assist in forestry planning and monitoring, the Ministry should: 


° produce the required forest information manual to establish policies regarding the 
collection of forest resources inventories; and 


* implement adequate oversight procedures to ensure that forest inventory information is 
accurate, publicly available, and up-to-date for each forest management unit. 


Current Status 


The Forest Information Manual, as previously noted, was completed and approved by 
regulation on April 25, 2001. This manual prescribes policy direction regarding the 
creation, maintenance, and provision of inventories for forest resources; and it describes a 
detailed process for companies to provide information in a timely fashion and in a standard 
and consistent format. With respect to the information being provided, the manual outlines 
required forest information, standards for data composition, and the roles and 
responsibilities of the Ministry and the forest management companies for the development 
and provision of access to forest information. 


In addition, the Ministry informed us that a review was carried out on the strategic 
direction of the forest resources inventory program, and the Ministry is implementing 
the key recommendations from this review. One of the key recommendations is that 
the Ministry maintain control and oversight responsibility for the following program 
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functions: co-ordination, the setting of policy and standards, quality control, quality 


assurance, and training and development. 


Planning And Harvesting 


Recommendation 


To help ensure that forest management units are managed on a sustainable basis and harvest 
operations are carried out in accordance with approved plans, the Ministry should require: 


¢ a thorough quantitative analysis of all substantial variations between planned and 
actual harvests; and 


© specific conclusions and recommendations for corrective action whenever there are 
significant variations from planned harvest volumes. 


Current Status 


‘The Ministry informed us that forest management companies are now required to submit 


actual harvest levels each year through the forest management unit’s annual report. In 
addition, forest management companies are required to prepare a quantitative analysis of 
any significant variations between planned and actual harvest levels based on their five-year 
forest management plans. This analysis is to include conclusions and recommendations for 
action that are to be incorporated into the next five-year forest management plan. 


In addition, any recommendations from independent audits relating to significant variances 
between planned and actual harvests are to be addressed by the forest management 
companies and the Ministry. In each case, an action plan is to be prepared that includes 
specific commitments and timelines to address the recommendations. 


Forest Resource Processing Facilities 


Recommendation 


Io promote sustainable forest management practices and optimize economic opportunities, the 


Ministry should ensure that: 


° all operating mills are licensed, as required by the Crown Forest Sustainability Act; 


the required annual information reports from mills are complete and received on a 
timely basis; and 


forest management company plans consider market demand as well as the capacities 
and requirements of the local mills. 


Current Status 


The Ministry informed us that it has implemented new procedures for the application for, 
and review and issuance of, licences for forest resource processing facilities. The Ministry is 
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also investigating a more streamlined approach to deal with company mergers and 
acquisitions—which were a major cause of facilities operating without a current licence. As 
well, licences can now be issued for one to five years (compared to the previous one-year 
licences). 


With respect to annual information reports from mills, the Ministry informed us that 
regional forestry staff work closely with district staff to ensure that all licensed facilities 
submit their annual returns within the timeframe required by the terms and conditions of 
their licences. In addition, towards the end of the 2002/03 fiscal year, the Ministry expects 
to begin allowing mills to apply for licences and submit their information reports using 
electronic means. 


The Ministry informed us that it will assess forest management plans to ensure they include 
and reflect a consideration of projected market demand for wood and the capacities and 
requirements of local mills. 


Forest Renewal and Maintenance 


Recommendation 


To ensure that all forestry companies manage their units on a sustainable basis and achieve 
sufficient renewal success, the Ministry should: 


° require companies to identify, explain and report on major variations between actual 
and planned renewal and maintenance activities; and 


° identify units where the renewal and maintenance work consistently falls short of target 
levels and implement corrective action to help achieve renewal success. 


Current Status 
The Ministry indicated that the Forest Management Planning Manual requires that planned 


and actual renewal activities be documented in forest management plans, annual work 
schedules, and management unit annual reports. Ministry staff will now assess the level of 
reported renewal and maintenance activities and compare this level to corresponding plans. 
When there are major discrepancies, the Ministry will require forest management 
companies to provide explanations along with an action plan for addressing the shortfalls. 


The Ministry also informed us that it continues to rely on the results of the independent 
audits to assess how well the forests are being renewed and the level of renewal survey 
activities being carried out. Any significant problems that are noted must be addressed with 
an action plan prepared by ministry and company staff. Subsequently, a two-year progress 
report is to be prepared that reports on the implementation status of the required corrective 
action. 





Porest Management Program Sus) 





| Follow-up Section 4.13 | 





i olewue Section 4.13 | 





COMPLIANCE MONITORING 


Compliance Inspections 


Recommendation 


To help ensure compliance with the Crown Forest Sustainability Act through an effective and 
economical inspection process, the Ministry should: 


¢ identify areas at high risk of non-compliance where ministry inspection staff should 
focus their efforts; 


° provide forest management companies with information based on previous ministry 
experience and analysis of trends in violations and non-compliance to help them 
improve their forest compliance plans; 


¢ where necessary, monitor and upgrade the skills of forest management company 
inspectors; and 


° assess the inspection process as it is currently operating and resolve any concerns that 
have resulted from the transfer of inspection responsibilities to the industry. 


Current Status 


The Ministry informed us it was in the process of enhancing the risk-based assessment 
process in its compliance, planning, and monitoring programs. For instance, the Ministry 
now requires that district compliance plans prioritize areas for mandatory spot checks and 
audits by qualified compliance inspectors. As well, the Ministry will annually review and 
approve the forest industry compliance plans to ensure that areas at higher risk of 
non-compliance are identified for inclusion in the forest management company’s inspection 
and reporting procedures. 


According to the Ministry, it is committed to providing forest management companies with 
information that is based on its own inspection experience. During the development and 
approval of company compliance plans, the Ministry’s district office staff will identify trends 
in non-compliance and violations to ensure that the company’s inspection procedures 
address problems noted in past inspections. 


Finally, in 2000/01, the Ministry implemented a competency-training program for ministry 
and industry inspectors and intends to implement mandatory testing and certification 
standards for its staff. (The training program is not mandatory for industry staff.) The 
Ministry indicated that the vast majority of industry inspectors participating in the training 
courses have opted for certification. The training and certification processes are designed to 


raise the skill levels of ministry and industry inspectors and assist in enhancing the accuracy 
and consistency of the inspection process. 
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Enforcement 


Recommendation 


10 improve overall compliance with the Crown Forest Sustainability Act and protect Ontario’ 
forests from activities that could impair forest sustainability, the Ministry should: 


° implement procedures to ensure that the information recorded in the compliance system 
is complete and can be used for management decision-making purposes, such as 


identifying and tracking repeat offenders; 


° review the enforcement approaches used by district offices to ensure that penalties are 
imposed in a consistent manner; and 


° analyze the extent of specific violations to identify trends and areas where more severe 
penalties or other corrective action may be required. 


Current Status 


The Ministry informed us it is currently reviewing the functionality of all of its compliance 
reporting systems. 


In addition, the Ministry is examining the mechanisms used to track trends and repeat 
offenders as well as monitoring for consistency in the application of penalties. For example, 
historical enforcement information will be used where data is available to compare offences 
noted and penalties imposed by different ministry district offices. 


FORESTRY TRUSTS 


Recommendation 


To ensure that the Forest Renewal and Forestry Futures trusts are administered in accordance 
with trust agreements, policy, and legislation, the Ministry should: 


¢ monitor Forest Renewal Trust account balances to ensure that each forest management 
company maintains the minimum balance required so that funds will be available for 
renewal activities when needed; 


* ensure that forestry charges are transferred to the Forestry Futures Trust on a timely 
basis; and 


* ensure that payments from the Forest Renewal Trust are made only after the required 
final invoice report is received. 


Current Status 


The Ministry informed us that staff now track revenue and expenditures related to the 

Forest Renewal Trust. The monthly account statements are reviewed and compared to the 
minimum balances. Quarterly summaries are provided to district staff, and when accounts 
contain less than the minimum balance, companies are contacted to correct the deficiency. 
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In April 2000, to ensure more timely transfers between trust accounts, the Ministry and the 
trustee changed the process so that Crown charges are now transferred directly to the 
Forestry Futures Trust. In addition, final payments from the Forest Renewal ‘Trust are to be 
made only after the required final invoice report is received. This requirement has been 
reinforced with the trustee and compliance with it will be monitored by the Ministry. 


FORESTRY REVENUE 


Recommendation 


To ensure that the province receives the proper amounts of revenue, the Ministry should 
systematically address any recurring problems noted through its audit examinations of stumpage 
charges. 


Current Status 


The Ministry informed us that it continues to carry out audits for timber measurement and 
related stumpage charges for 15 to 20 companies a year. Any company where problems 
with bills of lading have been noted through audits will be subject to a follow-up inspection 
within 30 days. In addition, new computer software was introduced in the Timber 
Resources Evaluation System to track bills of lading and to use the information for 
verification and audit purposes. 


With respect to private wood—which is not subject to stumpage charges—the Ministry 
implemented new procedures that place the onus on the private landowner to remit and 
the receiving forest company to obtain proof of private land ownership. 
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4.14—Ontario Native Affairs Secretariat 


(Follow-up to VFM Section 3.14, Special Report on 
Accountability and Value for Money—2000) 





BACKGROUND 


The Ontario Native Affairs Secretariat works with First Nations, Aboriginal organizations, 
and businesses to build strong, prosperous, and self-reliant Aboriginal communities. The 
Secretariat conducts land-claim negotiations on behalf of the province, implements land- 
claim settlements, provides core and capital funding for Aboriginal organizations and 
projects, and fosters Aboriginal economic development. Expenditures directly managed by 
the Secretariat for the 1999/2000 fiscal year were $18.7 million, and comparable 
expenditures for the 2001/02 fiscal year were $16.2 million. 


In our Special Report on Accountability and Value for Money (2000), we found that although 
the value of land claim settlements was adequately supported by independent evaluations or 
other evidence, improvements were needed in the timeliness of reporting and accountability 
by First Nations for the use of funding provided to them for land claim negotiation costs. 


For the two capital programs funded by the Secretariat but delivered by two other 
ministries, we found that secretariat monitoring was inadequate and that the arrangement 
whereby the Secretariat funded but ministries delivered these programs blurred the lines of 
accountability. 


In its advisory role, the Secretariat helps co-ordinate Aboriginal-specific programs delivered 
by other ministries. Expenditures for these programs exceeded $370 million annually. We 
concluded that the Secretariat needed to improve the timeliness of, accessibility to, and level 
of detail in its database of information on these programs. 


Accordingly, at that time we made a number of recommendations for improvement and 
received commitments from the Secretariat that it would take corrective action. 


CURRENT STATUS OF RECOMMENDATIONS 


Based on information obtained from the Secretariat and our review of this documentation, 
the Secretariat has been proactive in addressing our recommendations and some have been 
substantially implemented. The current status of the Secretariat’s action on each of our 
recommendations is as follows. 
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LAND CLAIMS 


Negotiation Funding 


Recommendation 


To improve accountability for the funding of Aboriginal land claim negotiation costs, the 
Secretariat should: 


° consider holding back a percentage of each year’s funding until it has received the 
required final reports and audited financial statements for that year; 


¢ establish a standard format for financial reports and audited financial statements to 
facilitate analysis of significant variances from budget and identification of ineligible 
expenditures and funding surpluses; and 


¢ fund projected negotiation costs only through the contribution agreement process. 


Current status 


Since our 2000 audit, the Secretariat issued a new negotiation funding guideline, which 
introduced a provision to hold back 15% of the annual funding, to be released once the 
Secretariat has reviewed and approved the required final financial reports and the audited 
financial statements. 


The Secretariat has also developed new standardized reporting requirements for annual 
budget submissions, financial reports, and audited financial statements. These new 
requirements have been incorporated into the 2001/02 contribution agreements. The 
required formats for the financial reports and the audited financial statements were also 
outlined in the contribution agreements. 


Finally, in March 2001, the Director of the Secretariat’s negotiation branch issued a memo 


to all negotiators, directing them to make every effort to avoid paying projected negotiation 
costs as part of a settlement. 


Land-claim Costing 


Recommendation 


1o assist with the evaluation of future land claims and enhance accountability for the cost- 
offectiveness of the program, the Secretariat should identify and track the total cost of each land 


claim. 


Current status 


In addition to tracking transfer payments and other costs, such as professional services, for 
each individual land-claim negotiation, in 2001/02 the Secretariat began tracking the cost 
of the time worked on each land claim by negotiators and legal support staff. 
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PROGRAMS ADMINISTERED BY OTHER MINISTRIES 


Recommendation 


In order to eliminate unnecessary overlap of monitoring and administrative activities, the 
Secretariat should in future strive to avoid taking on funding responsibility for programs that 
more appropriately could be administered by other ministries. 


While the current funding arrangements remain in place, the Secretariat needs to improve its 
monitoring of these programs in order to know whether they are functioning as intended or need 
corrective action. 


Current status 


We were advised by the Secretariat that, since our audit, it has not taken any new funding 
responsibilities for programs that could more appropriately be administered by other 
ministries. 

The Secretariat signed a new partnership agreement with the Ministry of Northern 
Development and Mines (MNDM) specifying new standards for it to report progress on 
the Canada-Ontario Infrastructure Agreement, which MNDM administers for the 


Secretariat. 


CORE FUNDING FOR ELIGIBLE ABORIGINAL 
ORGANIZATIONS 


Recommendation 


1o improve the budgeting process and the monitoring of expenditures for the Aboriginal 
Organizations Core Funding Program, the Secretariat should ensure that: 


° budget submissions are sufficiently detailed to enable the Secretariat to provide funding 
commensurate with the demand for and value of the services to be provided: and 


° all management letters are obtained, relevant issues identified, and significant control 
deficiencies are followed up on a timely basis. 


Current status 


The three organizations that received core funding were informed in writing in June 2001 
that budget submissions to the Secretariat had to provide more details on specific plans, 
projects, and anticipated outcomes. In addition, the Secretariat issued an “addendum 
regarding additional requirements for the 2001-2002 budget submission to Ontario’s 
Aboriginal Organizations Core Funding Program” that specified the information to be 
included in the budget submissions. 


However, core funding is still historically based. Each organization receives the same amount 
of funding from year to year, regardless of its plans or expenditure patterns. We were 
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informed that the Secretariat is planning a program evaluation of the core funding program 
that will include an examination of the basis for funding levels. 


In cases where significant control deficiencies are identified, the Secretariat requests and 
receives response letters from the core funding recipients that outline the steps they are 


taking to address the deficiencies. 


CO-ORDINATION OF ABORIGINAL PROGRAMS 


Information for Decision Making 


Recommendation 


To improve the usefulness of its database as a tool for government-wide co-ordination of 
provincial programs and services with expenditures of over $370 million delivered to Aboriginal 
residents, the Secretariat should: 


© obtain information on program services and results in addition to the financial infor- 
mation it currently collects; 


° integrate the information obtained on the Building Aboriginal Economies Strategy 
and the Working Partnerships Program into the database; and 


¢ take steps to make the database directly accessible to program managers in the 16 
ministries delivering programs and services to Aboriginal residents. 


Current status 


The Secretariat raised the database improvement issue with the Deputy Ministers’ 
Committee on Aboriginal Issues. However, the committee concluded that there was 
insufficient benefit to justify the collection of additional information on program services 
and results for the Aboriginal expenditure database. 


The Secretariat advised us that it will not include output-based information from the 
Building Aboriginal Economies Strategy in the expenditure database. However, it has 
begun tracking expenditures of participating ministries and combining this information 


with the results achieved from the Building Aboriginal Economies Strategy and the 
Working Partnerships Program. 


The Secretariat informed the chief administrative officers of all ministries with Aboriginal 


expenditures in writing that the Aboriginal expenditure database information is available to 
program managers. 
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Building Aboriginal Economies Strategy 


Recommendation 


In order to properly assess the performance of the Building Aboriginal Economies Strategy, the 
Secretariat should ensure that reported results actually arise from Building Aboriginal Economies 
Strategy initiatives. 


Current status 


The Secretariat hired two consulting groups to review and assess the Strategy’s data tracking 
systems at participating ministries. The consultants expressed concerns that were similar to 
those we expressed in our 2000 report regarding the accuracy of performance data for the 
Strategy that is collected by the Secretariat. One consultant made specific recommendations 
to improve four ministries’ systems. The Deputy Ministers’ Committee on Aboriginal Issues 
requested that the four ministries notify the Secretariat by January 11, 2002 as to whether 
they would implement the specific consultant’s recommendations relating to their systems. 
At the time of our follow-up, two ministries had still not notified the Secretariat, We were 
informed that they subsequently complied with the request. 


eee 


Ontario Native Affairs Secretariat 407 





| Follow-up Section 4.14 | 





| Follow-up Section 4.15 | 





MINISTRY OF TRANSPORTATION 


4.15—Monitoring School Purpose Vehicle 
Safety 


(Follow-up to VFM Section 3.15, Special Report on 
Accountability and Value for Money—2000) 





BACKGROUND 


In conjunction with our audit of pupil transportation grants provided to school boards by 
the Ministry of Education, we determined that it was important to also consider the 
Ministry of Transportation’s role in ensuring that pupil transportation is safe. 


We concluded that the Ministry could and should strengthen its systems and procedures for 
ensuring that operators of school purpose vehicles comply with legislative and regulatory 
safety requirements. In particular, the Ministry had not captured the information needed to 
ensure that: 


¢ all school buses were subject to being selected for inspection; and 


¢ those operator facilities and inspection stations posing the highest risk of non- 
compliance were selected for audit. 


We also concluded that the Ministry had not sufficiently communicated the nature, extent, 
and results of its enforcement activities to school boards and needed to co-ordinate efforts 
with them so that all safety risks are addressed and appropriate actions taken. 


We made several recommendations for improvement and received commitments from the 
Ministry that it would take corrective action. 


CURRENT STATUS OF RECOMMENDATIONS 


Based on the information provided to us by the Ministry of Transportation, the Ministry has 
taken some action to implement each of the recommendations made in our Special Report 


on Accountability and Value for Money (2000). The current status of each of our 
recommendations is outlined below. 


STRENGTHENING ENFORCEMENT PROCEDURES 


Recommendation 


10 obtain reasonable assurance that school bus operators are complying with legislative and 
regulatory requirements, the Ministry should: 
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° establish assurance objectives for its audit/inspection program and determine resource 
requirements based on these objectives; 


° document and periodically assess whether its audit and inspection methods are 
appropriate and sufficient; 


* make greater use of information systems technology to better focus audit and inspection 
activities on high-risk operators; 


° establish reporting and monitoring procedures that allow management to ensure 
enforcement personnel throughout the province are conducting audit/inspection 
activities tn accordance with the Branch’s policies; and 


° establish follow-up procedures to verify that school bus operators take timely action to 
correct problems detected by audits of facilities and Motor Vehicle Inspection Stations. 


Current Status 


The Ministry advised us that it had addressed our recommendations through revisions to its 
audit/inspection procedures, improvements in management review and results-reporting 
procedures, and the development of a Bus Information Tracking System (BITS). The 
Ministry expected to complete the input of detailed information about each bus operator 
and each vehicle in the operator’s fleet by the end of 2002. Enforcement officers are to 
input the results of their enforcement activities into the BITS starting in September 2002. 


The Ministry was expecting that by mid-2004 the BITS would contain a sufficient history 
of enforcement results to enable the Ministry to utilize the system to help identify and target 
high-risk operators and to contribute to earlier intervention by enforcement officers where 
warranted. Until that time, the Ministry advised us that its enforcement staff will continue to 
target high-risk operators using local knowledge of operators in their area. 


ESTABLISHING A PROTOCOL FOR CO-OPERATION WITH 
SCHOOL BOARDS | 


Recommendation 


To better ensure that timely action is taken to enforce pupil transportation safety requirements, 
the Ministry should work with school boards to develop a protocol that sets out the expectations of 
each party and establishes procedures to co-ordinate activities and exchange the results of their 
work. 


Current Status 


The Ministry had developed, in co-operation with the Ontario Association of School 
Business Officials, a protocol that, when fully implemented, would address our 
recommendation. This protocol has been shared with the Ontario School Bus Association 
and is intended to serve as an effective deterrent to the operation of unsafe school purpose 
vehicles. The Ministry was expecting that the protocol would be in force starting in the 
2002/03 school year. 
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CHAPTER FIVE 


Public Accounts of the 
Province 





INTRODUCTION 


The Public Accounts for each fiscal year, ending March 31, are prepared under the 
direction of the Minister of Finance as required by the Ministry of Treasury and Economics 
Act. The Act requires the Public Accounts to be delivered to the Lieutenant Governor in 
Council for presentation to the Legislative Assembly not later than the tenth day of the first 
session held in the following calendar year. The Public Accounts are typically tabled in the 
early fall of each year. 


The financial statements of the province, which are included in the Public Accounts, are the 
responsibility of the Government of Ontario. This responsibility encompasses ensuring the 
integrity and fairness of the information presented in the statements, including the many 
amounts based on estimates and judgment. The government is also responsible for ensuring 
that an established system of control with supporting procedures is in place to provide 
assurance that transactions are authorized, assets are safeguarded, and proper records are 
maintained. 


I audit and express an opinion on the financial statements of the province. The objective of 
my audit is to determine, with reasonable assurance, whether the financial statements are 
free of material misstatement—that is, that they are free of significant errors or omissions. 
The financial statements, along with my Auditor’s Report on them, are included in the 
province's annual report and provided in a separate volume of the Public Accounts. 


In addition to the financial statements, the province’s annual report presents summaries and 
analyses of the province’s financial condition and fiscal results. As such, it enhances the fiscal 
accountability of the government to both the Legislative Assembly and the public. 


The Public Accounts also include three supplementary volumes: 


Volume 1 contains the Consolidated Revenue Fund schedules and ministry statements. 
These schedules and statements reflect the financial activities of the government s 
ministries on a modified cash basis of accounting. 


Volume 2 contains the financial statements of significant provincial Crown corporations, 


boards, and commissions that are part of the government's reporting entity, and other 


miscellaneous financial statements. 
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* Volume 3 contains further details of public expenditures as well as the Ontario Public 
Service senior salary disclosure. 


I review the information in the annual report and the three supplementary volumes for 
consistency with the information presented in the financial statements. 


Legislative changes to the Act have been made with respect to the Public Accounts 
reporting process. Effective April 1, 2003, the Public Accounts will be redefined to consist 
of an Annual Report that combines the existing annual report and the financial statements. 
This combined report, except in extraordinary circumstances, is to be delivered to the 
Lieutenant Governor in Council on or before the 180th day after the end of the fiscal year. 
The Lieutenant Governor in Council must then either lay the Public Accounts before the 
Assembly, or, if the Assembly is not in session, make the Public Accounts public and lay 
them before the Assembly on or before the tenth day of the next session. 


With regard to the three supplementary volumes, except in extraordinary circumstances, 
these must be submitted to the Lieutenant Governor in Council before the 240th day after 
the end of the fiscal year. The Lieutenant Governor in Council must then lay the 
information before the Assembly, or, if it is not in session, lay it before the Assembly on or 
before the tenth day of the next session. 


THE PROVINCE’S 
2001/02 FINANCIAL STATEMENTS 


The Audit Act requires that in my Annual Report I report on the results of my examination 
of the province's financial statements as reported in the Public Accounts. I am pleased to 
report that my Auditor’s Report to the Legislative Assembly on the financial statements for 
the year ended March 31, 2002 is clear of any qualifications or reservations and reads as 
follows: 


To the Legislative Assembly of the Province of Ontario 


| have audited the statement of financial position of the Province of Ontario as at 
March 31, 2002 and the statements of revenue, expenditure, and net debt and of 
cash flows for the year then ended. These financial statements are the 
responsibility of the Government of Ontario. My responsibility is to express an 
opinion on these financial statements based on my audit. 


| conducted my audit in accordance with Canadian generally accepted auditing 
standards. Those standards require that | plan and perform an audit to obtain 
reasonable assurance whether the financial statements are free of material 
misstatement. An audit includes examining, on a test basis, evidence supporting 
the amounts and disclosures in the financial statements. The audit also includes 





Public Accounts of the Province 41] 


| 


Chapter Five 





| 


Chapter Five 





ee _____._______ | | 


assessing the accounting principles used and significant estimates made by the 
Government, as well as evaluating the overall financial statement presentation. 


In my opinion, these financial statements present fairly, in all material respects, the 
financial position of the Province as at March 31, 2002 and the results of its 
operations and its cash flows for the year then ended in accordance with accounting 
principles recommended for governments by the Canadian Institute of Chartered 
Accountants. As required by Section 12 of the Audit Act, | also report that, in my 
opinion, after the restatements explained in Note 2 to the financial statements, 
these accounting principles have been applied, in all material respects, on a basis 
consistent with that of the preceding year. 


[signed] 
Toronto, Ontario Erik Peters, FCA 
September 10, 2002 Provincial Auditor 


THE GOVERNMENT 
REPORTING ENTITY 


One of the most critical aspects of reporting on a government's financial affairs is deciding 
which entities—including agencies, Crown-controlled corporations, boards, commissions, 
and other organizations receiving transfer payments—should be included in the 
government's financial statements. Inclusion essentially means that an entity’s operating 
results and its assets and liabilities are consolidated with or otherwise incorporated into the 
government's financial statements and form part of the government’s annual surplus or 
deficit and its net assets or liabilities. 


Currently, the Ontario government prepares its financial statements in accordance with the 
accounting principles recommended for governments by the Public Sector Accounting 
Board (PSAB) of the Canadian Institute of Chartered Accountants. A key objective of 
PSAB's standards, particularly its reporting entity standard, is to assist governments in 
preparing financial statements that account for the full nature and extent of the financial 
affairs and resources for which it is responsible. The current reporting entity standard 
recommends that entities should be included in the government's financial statements if: (1) 
they are accountable for the administration of their financial affairs and resources either to a 


minister of the government or directly to the Legislature and (2) they are owned or 
controlled by the government. 


In following PSAB’s reporting entity standard, Ontario’s financial statements specifically 
include, in addition to all government ministries, the activities of twenty-seven of its most 
significant organizations, such as Ontario Power Generation Inc., Hydro One Inc., the 
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Ontario Electricity Financial Corporation, the Ontario Lottery and Gaming Corporation, 
the Liquor Control Board of Ontario, GO Transit, and the Ontario Housing Corporation. 
Less significant government organizations are reflected in the financial statements through 
the accounts of the ministries responsible for them. 


PSAB’s current reporting entity standard is being differently interpreted and applied 
throughout Canadian jurisdictions, particularly among provincial governments. Key to 
these different interpretations is whether the reporting entity standard currently does 
include, or should be amended to include, a number of significant public-sector or quasi- 
public-sector institutions, most of which are primarily funded by provincial governments 
but operate outside of the government ministry and agency structure. Historically, the 
majority of these institutions have often been collectively referred to as forming the SUCH 
sector (for school boards, universities, colleges, and hospitals). Another area of debate is 
special-purpose agencies, such as trusts or foundations that have recently been created in 
some jurisdictions, typically to act as financial intermediaries. These agencies are often 
granted considerable sums of money by the government under a mandate to eventually pass 
on these funds via grants or transfers, often years later, to the actual intended recipients to 
support specific government programs. 


In Ontario, SUCH-sector organizations are not included in the reporting entity because 
they are not considered to have met PSAB’s inclusion criteria of accountability and 
ownership or control. Ontario also does not include in its reporting entity one major special- 
purpose agency, the Ontario Innovation Trust (the Trust). The Trust has received and is 
administering large sums of government money outside of the reporting entity. I have raised 
accounting and accountability concerns with respect to the Trust in this and in each of my 
last three reports. 


With respect to Ontario’s SUCH-sector institutions, many of these organizations are funded 
primarily, if not entirely, by the Ontario government and are economically dependent on it. 
Many are also subject to varying degrees of provincial control over their operations. These 
include many of Ontario’s health-care organizations (for instance, hospitals, long-term-care 
facilities, community health centres, and community care access centres), its school boards, 
its colleges, and its universities. It should be noted that many of these institutions have 
considerable assets, liabilities (including long-term debt), revenues, and expenditures. If such 
agencies were included in the government's financial statements, it would have a significant 
impact on the province’s reported financial position and its operating results. 


This issue can perhaps best be presented through an example. In recent years, it has become 
increasingly clear that the government has significantly increased its control over the delivery 
of Ontario’s elementary- and secondary-school programs. The provincial government, 
rather than local boards, now sets the local education tax rate. Such education taxes, raised 
from the local property tax base, are no longer provided directly to the local school boards 
for use in local schools. Rather, the Ontario government now redistributes such taxes 
throughout the province on a centrally determined basis. The government has further 
tightened controls over school-board operations by establishing legislation and policies in 
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such key areas as allowable average class sizes, hours of instruction, province-wide curricula, 
and student testing. In response to projected illegal operating deficits, the government 
recently further increased its control by directly taking over three of the province’s largest 
school boards. In effect, the Ministry of Education now centrally manages and controls the 
most significant aspects of the primary and secondary education system. 


Despite these changes, school boards remain outside of the Ontario government's reporting 
entity. Local education taxes, which are effectively controlled by the Ontario government 
and amount to about $6 billion annually, are not included in Ontario’s reported revenues. 
The actual education costs incurred by the boards in delivering education programs to 
Ontarians are also not included in the province’s expenditures. Only transfer payments 
made directly by the province to school boards are reflected. School board capital debts, 
which are considerable, are not included as part of provincial debt even though the 
government has made a long-term commitment to fund these debt repayments. In 
addition, the school boards’ capital assets and other assets and liabilities are not included in 
the province’s financial statements. 


To date, I have accepted the disclosure of grants to school boards as education expenditure 
as the fairest presentation in the financial statements of the province because the province 
has not yet adopted the PSAB accounting standards for recording tangible capital assets. 
Under current provincial accounting practices, if the financial statements of school boards 
were included, only the school boards’ debt would be shown, not the value of the schools 
built with that debt. I would not consider that omission to result in a better financial 
presentation than the March 31, 2002 disclosure of grants to school boards as education 
expenditure. This situation is expected to change in the current fiscal year. 


Effective with the financial statements for the year ending March 31, 2003, the province 
intends to adopt new PSAB accounting rules for tangible capital assets. One effect of this 
decision would be the ability to eventually fully consolidate the financial statements of school 
boards in the financial statements of the province—that is, when school boards’ accounting 
practices for tangible capital assets allow this to be done. I urge that this financial 
consolidation be considered given the control exercised by the Ministry of Education over 
school boards. (PSAB’s concept of “control” is discussed below.) 


Although I have focused my discussion on school boards, similar reporting entity issues exist 
for other SUCH-sector institutions in the province. As a result of continuing concerns 
expressed throughout Canada about the existing reporting entity definition and problems 
with its interpretation and application, PSAB recently released a discussion paper entitled 
“Government Reporting Entity Discussion Paper”. PSAB’s current intention is that the 
principles outlined in this discussion paper will form the basis for a new reporting entity 
standard, expected in early 2003. The discussion paper focuses exclusively on control as the 


determining factor for inclusion. The previous accountability and ownership criteria would 
be subsumed under this control umbrella. 
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The discussion paper provides a proposed definition of control for governments and 
outlines indicators of control in considerable detail. It further proposes that the substance of 
the organization’s relationship with the government be the key determinant for inclusion or 
exclusion from the reporting entity. The paper proposes three elements to the definition of 
control. First, regardless of whether control is being exercised or not, the power to control is 
the key to determining whether entities should be included. Second, for control to exist, the 
government must be able to direct the financial and operating policies of the organization. 
Third, the government must expect to benefit from the activities of the organization, or be 
exposed to loss. 


I urge the Ontario government to treat the release of PSAB’s discussion paper as an 
opportunity to make improvements in this important financial reporting area. It is 
particularly germane to re-examine this issue now as Ontario is moving (as discussed later in 
this chapter) to reflect its tangible capital assets in its financial statements. I recommend that 
the Ministry of Finance take immediate steps to initiate a review of its existing reporting 
entity. The review should encompass all the organizations in Ontario that are currently in 
the reporting entity as well as those currently outside of it. Special emphasis should be 
placed on assessing the legislative, regulatory, funding, and accountability relationships the 
government has in place with each SUCH-sector group or other organizations that receive 
major transfer payments, as well as any special-purpose agencies created by the Ontario 
government, such as the Ontario Innovation Trust. The results of the review should be 
updated as necessary to reflect the recommendations in PSAB’s final, issued reporting 
standard. 


MULTI-YEAR FUNDING 


In last year’s Annual Report, I reported concerns I had regarding the government’s 
accounting and accountability for multi-year funding. I illustrated my concern by detailing 
a number of problems with the approach taken by the government in accounting for 
Ontario's health-care expenditures, particularly the practice of charging to one fiscal period 
transfer payments covering the activities of several years. As I indicated last year, it is essential 
that the annual operating statements of government properly reflect revenues and 
expenditures relating to the fiscal period being measured. When this practice is not followed 
and distortions are significant, users of financial statements cannot evaluate a government's 
fiscal performance for the year vis-a-vis its budget, assess its revenues earned vis-a-vis its 
expenditures on government programs, or make useful comparisons of such information 
between past and future periods or between jurisdictions. 


This year I noted no further instances of multi-year funding being inappropriately charged 
to one fiscal period. 


The Canadian Institute of Chartered Accountants has recognized that the current 
standards for accounting for government transfers allow for considerable latitude and has 
established a task force to study this issue: the related project is called Government Transfers. 
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ADOPTING CONSISTENT ACCOUNTING RULES 
FOR ALL FINANCIAL REPORTS 


In my 2001 Annual Report, I stated that the fact that the province's accounting system 
continued to be maintained and publicly disclosed on two separate bases contributed to the 
lack of transparency, and I used as an example the inconsistency in Ontario's accounting for 
health-care expenditures. Specifically, I noted that $1 billion of capital funding for hospitals 
was reported in the province’s financial statements as health-care expenditure for the year 
ended March 31, 2000, but in Volume 1 of the Public Accounts, the $1 billion was 
reported as a health-care expenditure for the year ended March 31, 2001. This year I note 
that corporation tax revenue is shown as $7.5 billion in Volume 1 of the Public Accounts, 
but it is shown as $6.6 billion in the province’s financial statements for the year ended 


March 31, 2002. 


The modified-cash basis of accounting is used for legislative appropriation control and 
Volume 1, and the modified accrual basis of accounting as prescribed by PSAB is used for 
purposes of the province’s financial statements. I urged the adoption of the accrual 
accounting approach for all of the government’s public financial reports. 


This year the government introduced legislation, Keeping the Promise for Growth and 
Prosperity Act (2002 Budget), 2002, to convert Ontario’s legislative spending authority and 
appropriation control to the accrual basis of accounting. Effective April 1, 2003, the 
government intends to use the accrual basis of accounting in the preparation of all 
government spending estimates and for the reporting of actual fiscal results against these 
estimates in Volumes 1 and 3 of the Public Accounts. This step will significantly enhance the 
public accountability of the government. 


FEDERAL TAX ERROR 


The Canada Customs and Revenue Agency (CCRA) collects federal and provincial 
personal income taxes on behalf of all provinces except Quebec. On January 29, 2002, 
CCRA advised the provinces that it had made an error in processing personal income taxes 
going back to 1972, which had resulted in significant overpayments to several of the 
provinces. The error related to capital gains refunds that had been credited by CCRA to 
mutual fund trusts. A portion of the refunds credited should have been deducted from the 


personal income tax amounts remitted to certain of the provinces over the years but this was 
never done. 


Although the error had likely been occurring since 1972, supporting data was only 
available back to 1993. The extent of the federal error relating to the 1993-99 period was 
estimated to be approximately $3.3 billion, $2.8 billion of which applied to Ontario. In 
early 2002, CCRA was in the process of finalizing the actual amount of the error and, as 
part of this work, had asked the Auditor General of Canada to undertake audit procedures 


to verify the amount of the error once it had been fully determined. 
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To provide additional assurances to the provinces regarding the work of the federal Auditor 
General’s office, a working group consisting of senior representatives from five provincial 
legislative audit offices—Ontario, Manitoba, British Columbia, Alberta, and Quebec—was 
established to review the planned work of the Auditor General, provide input on any issues 
arising during that work, and assess the impact on the provinces of the work done. My staff 
liaised with staffin Ontario’s Ministry of Finance throughout the course of our work on this 
issue. 


Several meetings between the federal Auditor General's office and representatives from the 
provincial audit offices were held in March, April, and May 2002. The Auditor General’s 
office was very receptive to suggestions and carried out certain additional audit work at the 


request of the provinces. 


On May 31, 2002, the Auditor General of Canada issued four reports, two of which 
related to the errors for 1993-99: 


¢ An Auditor's Report wherein the Auditor General concluded that CCRA’s Statement 
of Assessed Mutual Fund Trust Provincial Capital Gains Refunds (the Statement) for the 
1997-99 tax years was presented fairly. This Statement showed the amount of the error 
affecting Ontario for those years as being $2.074 billion. 
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¢ An Accountant's Report relating to a similar Statement for the earlier 1993-96 tax 
years, which showed the amount of the error affecting Ontario for those years as being 


$738 million. 


An Accountant's Report does not constitute an audit, and while it outlines the specific 
procedures performed by the Auditor General, it does not provide the same high level 
of assurance as that provided in the aforementioned Auditor’s Report. 


On September 4, 2002, the federal Deputy Prime Minister and Minister of Finance 
announced that the Government of Canada would recover $1.4 billion of the total 
amounts overpaid to the provinces for the years 1997-99, of which $1.330 billion related 
to Ontario. The Minister indicated that the recovery would be restricted to a portion of the 
total amount attributed to 1997-99, the period for which the Auditor General of Canada 
provided a high level of assurance. The federal government intends to recover the 

$1.330 billion over ten years, starting in the 2004/05 fiscal year, with no interest penalties. 


Accordingly, the province has made provisional adjustments in its 2001/02 financial 
statements to reflect the amount the federal government intends to recover for 1997-99. 
However, it is important to note that Ontario disagrees with the position taken by the 
Canadian government on its recovery. The province believes that the responsibility for the 
error lies with the federal government. As indicated in the province’s Annual Report, 
Ontario believes the conservative manner in which they have accounted for this error does 
not preclude them from in future taking action, including legal action, against the CCRA 
and/or the Canadian government. 
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As well, based on prior years’ actual federal payments, Ontario’s Ministry of Finance 
developed an estimate of what would be received from the federal government for the 2000 
and 2001 tax years relating to trust income tax and recorded the amount relating to the 
period April 1, 2000 to March 31, 2001 as a receivable and revenue in its 2000/01 
financial statements. However, when the federal error relating to mutual fund trusts became 
known, the Ministry provisionally recognized that $713 million of this recetvable would not 
be collectible and accordingly restated the 2000/01 amounts in its 2001/02 financial 


statements. 


The total impact of the federal tax error was retroactively recorded in the province's 
2001/02 financial statements with the 2000/01 balances being restated as outlined in the 
following table. 


Impact of Federal Error on Ontario’s 
2000/01 Financial Statements 


($ million) 


Provisional liabilities increase (for 1997-99) 1,330 





Provisional accounts receivable decrease (for 2000-01) 
Increase in net debt as at March 31, 2001 





* This resulted in a $713-million reduction in the previously reported 2000/01 
surplus. 


STRANDED DEBT OF THE ELECTRICITY SECTOR 


In my 2000 Annual Report, I addressed concerns relating to the recent restructuring of 
Ontario's electricity sector, particularly its financial impact on the government of Ontario. 
The most significant of these impacts was the identification and recognition on April 1, 
1999 of $19.4 billion of stranded debt to be recovered from ratepayers. This stranded debt 
arose because of a revaluation of Ontario Hydro’s assets to reflect their value in the new 
competitive environment. These values were significantly lower than the amounts that had 
previously been recorded in Ontario Hydro’s accounts. 


As I indicated in 2000, the government established a long-term plan to retire the stranded 
debt through dedicated revenue streams derived from the electricity sector. Since this debt is 
to be recovered from electricity ratepayers rather than taxpayers, it is separated from other 
government liabilities on the province’s statement of financial position, and the net annual 
results under the recovery plan are disclosed separately from the results of other government 
operations on the province’s statement of revenue, expenditure, and net debt. 


There are risks for Ontario’s taxpayers under the electricity restructuring arrangements. For 
instance, the recovery plan for the stranded debt is subject to considerable uncertainty 
because much of the revenue from which the stranded debt will be defeased is dependent 
on the future energy sales and profits of Ontario Hydro’s successor companies. These sales 
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and profits are difficult to predict because Ontario Hydro’s successor entities have just 
begun operating in a sector that the government opened to competition in May 2002. 


In my view, as of March 31, 2002, there is an increasing risk that part or all of the stranded 
debt will not be recoverable from the ratepayers and will therefore become a liability of the 
taxpayers. My view is based on the following observations: 


* The government's current plan indicates that the stranded debt should be defeased by 
the year 2012, but the projected defeasance date has been subject to revision. Whereas 
initially the government indicated that the stranded debt would be defeased as early as 
2010, the earliest projected defeasance date has now been delayed by two years. 





¢ The stranded debt has actually risen over the last three years, from $19.4 billion at 
April 1, 1999 to $20.1 billion at March 31, 2002. A portion of this increase 
($226 million) resulted from a change in accounting rules that required a retroactive 
restatement of Ontario Power Generation Inc.’s results. 
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¢ The financial performance of Hydro’s successor companies for the fiscal year ended 
March 31, 2002 was well below expectations. The government expected its two wholly 
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owned Hydro successor companies—Ontario Power Generation Inc. and Hydro One 
Inc.—to earn $524 million during that fiscal year; instead, the two corporations jointly 
earned only $179 million, a shortfall of $345 million. 


¢ The $179-million earnings in the 2001/02 fiscal year were insufficient to offset the 
government's $520-million annual interest charge on debt issued for its $8.9-billion 
investment in these two companies. It should be noted that, under the recovery plan, 
only amounts earned by the two successor companies above the $520 million are set 
aside for stranded debt recovery. Accordingly, none of the $179 million in earnings is 
available to reduce the stranded debt, and as a result $341 million was absorbed by 
Ontario’s taxpayers. 


Frequent and thorough reviews of the stranded debt recovery plan are required to ensure its 
continued viability. Since the government is responsible for the stranded debt, if its plan to 
recover the stranded debt through ratepayers fails, the taxpayer will ultimately be 
responsible for that debt. If the plan can no longer be supported, the government should 


recognize this debt on its financial statements as recoverable from the taxpayers. 


I also urge the government to include in all considerations of privatizations in the electricity 
sector a careful evaluation of the impacts of each privatization on the ratepayers’ ability to 
defease the stranded debt. For example, any net proceeds from privatization need to be 
assessed against the estimated future cash flow that will no longer be available to pay off the 
stranded debt. 


ACCOUNTING FOR TANGIBLE CAPITAL ASSETS 


Currently, Ontario ministries and government service organizations charge the full cost of 
capital assets to expenditures in the year of acquisition or construction. This differs from the 
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practice followed in the private sector, where capital assets acquired or constructed are 
initially recorded on the statement of financial position as assets and amortized and expensed 
to operations over their estimated useful lives. 


In June 1997, PSAB approved a new set of recommendations setting out rules for the 
recognition, measurement, amortization, and presentation of government capital assets. 
Among other things, the standard requires that a statement of tangible capital assets be 
included as part of a government's summary financial statements. The Ministry of Finance 
has not as yet adopted the recommendations contained in this standard. 


In December 1999, the government re-established the Ontario Financial Review 
Commission (Commission) to review the financial management practices of the government 
and its major transfer partners. Among the items the Commission examined were capital 
funding, capital financing, and options for reporting the government's investment in 
tangible capital assets. At the request of the Minister of Finance, and in conformity with 
Section 17 of the Audit Act, I served as special advisor to the Commission. 


The Commission issued its report last year. With respect to tangible capital assets, the 
Commission concluded that providing more information about the government's inventory 
of assets owned, the condition of those assets, and its plans for capital renewal, replacement, 
and disposal is essential. It noted that better reporting would give the public and 
government a better picture of the resources used to provide public services and should 
help managers within government make better decisions about how to invest in and 
maintain tangible capital assets. Its recommendations included the development of the 
information needed to show the cost and depreciation of existing tangible capital assets and 
the adoption of the existing PSAB standards for reporting tangible capital assets as soon as 
possible. 


In May 2002, PSAB issued an exposure draft entitled “Reporting Model—Senior 
Governments’, which proposed changes to the June 1997 standards on tangible capital 
assets. Among other things, the proposed model would require governments, in their 
statements of financial position, to report on both net debt and the accumulated surplus or 
deficit based on an expense basis of accounting. Under the expense basis of accounting, the 
statement of operations would include an annual charge amortizing the cost of tangible 
capital assets over their useful lives to arrive at a government’s reported surplus or deficit for 


the year rather than recording the full cost of assets acquired during the year as an 
expenditure. 


The government announced in its 2002 Budget its intention to adopt PSAB’s proposed 
recommendations relating to tangible capital assets beginning in fiscal 2002/03. Under a 
phased-in approach, the government will first value and record its investments in land, 
buildings, and the transportation infrastructure; and the remaining tangible capital assets 
owned by the province will be identified, valued, and incorporated into the statements in 
subsequent years. I concur with this phased-in approach, given that land, buildings, and the 


transportation infrastructure are estimated by the government to account for over 90% of 
its tangible capital assets. 
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There is little doubt that instituting a system to properly account for Ontario’s significant 
capital investments represents a challenge. I support both PSAB’s recommendations and 
those of the Commission: since 1994, I have stated my belief that summary financial 
statements reflecting the recommended enhanced financial information will be valuable for 
both government decision-makers and stakeholders. I am pleased with the steps being taken 
this year by the government in this area, and look forward to further consultations, to the 
extent possible while safeguarding my independence, in order that we may help ensure that 
existence, ownership, auditability, and valuation issues regarding capital assets are 
appropriately resolved. 
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OTHER RECOMMENDATIONS FOR 
IMPROVEMENT 


Although the audit of the province’s financial statements was not designed to identify all 
weaknesses in internal controls, nor to provide assurances on financial systems and 
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procedures as such, we noted a number of areas during the audit where we believed 
improvements could be made. While none of these matters affects the fairness of the 
financial statements of the province, they are covered, along with accompanying 
recommendations for improvement, in an annual management letter to the Ministry of 
Finance. 


NEW PSAB INITIATIVES 
The Public Sector Accounting Board (PSAB) of the Canadian Institute of Chartered 


Accountants attempts to foster improved government financial and performance 
information by continuously improving its existing recommendations and by developing 
new recommendations to deal with emerging accounting and auditing issues. The 
“Government Reporting Entity Discussion Paper”, the exposure draft entitled “Reporting 
Model—Senior Governments”, and the activities of the task force for the Government 
Transfers project have already been discussed in preceding sections of this chapter. In 
addition to these, some of the most significant issues PSAB is dealing with at the present 
time, from Ontario’s perspective, are the following: 


* PSAB recently approved a Statement of Principles for “Foreign Currency Translation”. 
This statement proposes that the current practice of deferral and amortization of gains 
and losses relating to foreign currency and assets and liabilities held in foreign currency 
be retained to recognize and measure the effects of changes in foreign exchange rates. 
This practice is currently already followed by the province. 


¢ PSAB recently approved a Statement of Principles for “Financial Statement Discussion 
and Analysis (FSD&A)”. The statement proposes principles and guidance for the 
presentation of FSD&A as required supplementary information in a government’s 
financial report. This supplementary information would include narrative explanations 
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and graphical illustrations of key events during the reporting period and explanations 
and illustrations of variances and trends. 


OTHER MATTERS 


The Provincial Auditor is required under section 12 of the Audit Act to report on any 
Special Warrants and Treasury Board Orders issued during the year. In addition, under 
section 91 of the Legislative Assembly Act, the Provincial Auditor is required to report on any 
transfers of money between Items within the same Vote in the Estimates of the Office of the 
Legislative Assembly. 


LEGISLATIVE APPROVAL OF GOVERNMENT 
EXPENDITURES 


The government tables detailed Expenditure Estimates, outlining each ministry's spending 
proposals on a program-by-program basis, shortly after presenting its Budget. The Standing 
Committee on Estimates reviews selected ministry Estimates and presents a report to the 
Legislature with respect to those ministry Estimates that were reviewed. The Estimates of 
those ministries that are not selected for review are deemed to be passed by the Committee 
and reported as such to the Legislature. Orders for Concurrence for each of the Estimates 
reported on by the Committee are debated in the Legislature for a maximum of six hours 
and then voted on. 


Once the Orders for Concurrence are approved, the Legislature provides the government 
with legal spending authority by approving the Supply Act, which stipulates the amounts 
that can be spent according to the ministry programs as set out in the Estimates. Once the 
Supply Act is approved, the individual program expenditures are considered Voted 
Appropriations. The Supply Act, 2001 pertaining to the fiscal year ended March 31, 2002, 
received Royal Assent on December 12, 2001. 


Typically, prior to the passage of the Supply Act, the Legislature authorizes payments by 
means of motions for interim supply. For the 2001/02 fiscal year, the time periods covered 
by the motions for interim supply and the dates that the motions were agreed to by the 
Legislature were as follows: 


* November 1, 2000 to April 30, 2001—passed September 27, 2000. 
¢ May 1, 2001 to October 31, 2001—passed April 23, 2001. 
¢ November 1, 2001 to April 30, 2002—passed October 22, 2001. 


SPECIAL WARRANTS 


If motions for interim supply cannot be approved because the Legislature is not in session, 
section 7 of the Treasury Board Act, 1991 allows the issue of Special Warrants authorizing 
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the expenditure of money for which there is no appropriation by the Legislature. Special 
Warrants are authorized by Orders in Council approved by the Lieutenant Governor on 
the recommendation of the government. 


As the three motions of interim supply covered the period from April 1, 2001 to March 31, 
2002, no Special Warrants were required during the 2001/02 fiscal year. 


TREASURY BOARD ORDERS 


Section 8 of the Treasury Board Act, 1991 allows the Treasury Board to make an order 
authorizing payments to supplement the amount of any Voted Appropriation that is 
insufficient to carry out the purpose for which it was made, provided the amount of the 
increase is offset by a corresponding reduction of expenditures from other Voted 
Appropriations not fully spent in the fiscal year. The order may be made at any time before 
the first day of May following the end of the fiscal year in which the supplemented 


appropriation was made. 


Chapter Five 





The following chart is a summary of the total value of Treasury Board Orders issued for the ered 
past five fiscal years: 
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Treasury Board Orders for the 2001/02 fiscal year summarized by month of issue are as 


follows: 


- Month of Issue pe “Number es eee 


Roe 2001—February 2002 231,023,300 


March 2002 675,194,600 


April 2002 eee | 1,450,000 | 
Total 907 667,900 








In accordance with a Standing Order of the Legislative Assembly, the preceding Treasury 
Board Orders are to be printed in The Ontario Gazette in the fall of 2002, together with 
explanatory information. A detailed listing of 2001/02 Treasury Board Orders, showing the 
amounts authorized and expended, is included as Exhibit Three of this report. 


TRANSFERS AUTHORIZED BY THE BOARD OF 
INTERNAL ECONOMY 


When the Board of Internal Economy authorizes the transfer of money from one Item of 
the Estimates of the Office of the Assembly to another Item within the same Vote, 

section 91 of the Legislative Assembly Act requires the Provincial Auditor to make special 
mention of the transfer(s) in the Annual Report. 


With respect to the 2001/02 Estimates, the following transfers were made within Vote 201: 


From: Item5 Administrative Services $ 178,200 
Item 6 Sergeant at Arms and Precinct Properties 34,500 
Item 8 = Caucus Support Services 1,700 
lO: Item 1 Office of the Speaker 700 
Item 2 Office of the Clerk 174,100 
Item 3 Legislative Services 28,400 
Item 4 Legislative Library 5,900 
Item 7 Legislative Information Systems 3,600 
Item 13 Restructuring Costs 1,700 


In addition, within Vote 202, $60,700 was transferred from Item 2 (Office of the 
Information and Privacy Commissioner) to Item 1 (Environmental Commissioner). 


UNCOLLECTIBLE ACCOUNTS 


Under section 5 of the Financial Administration Act, the Lieutenant Governor in Council, 
on the recommendation of the Minister of Finance, may authorize an Order in Council to 
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delete from the accounts any amount due to the Crown which is deemed uncollectible. The 
losses deleted from the accounts during any fiscal year are to be reported in the Public 
Accounts. 


In the 2001/02 fiscal year, receivables of $126.5 million due to the Crown from individuals 
and non-government organizations were written off (in 2000/01 the comparable amount 
was $152.9 million). Volume 2 of the 2001/02 Public Accounts of Ontario provides a listing 
of these write-offs in total by ministry or Crown agency. 


Under the accounting policies followed in the audited financial statements of the province, 
a provision for doubtful accounts is recorded against the accounts-receivable balances. 
Accordingly, most of the $126.5 million in write-offs had already been provided for in the 
audited financial statements. However, the actual deletion from the accounts required 


Order-in-Council approval. 

The major portion of the write-offs related to the following: 

* $47.1 million for uncollectible taxes relating to retail sales tax receivables; 

* $37.7 million for uncollectible taxes relating to corporation tax receivables: 

* $8.9 million for uncollectible taxes relating to employer health tax receivables; and 


* $6 million for uncollectible assessments under the Ontario Disability Support Program. 
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CHAPTER SIX 


The Office of the 
Provincial Auditor 











MISSION STATEMENT 


Our mission is to report to the Legislative Assembly objective information and 
recommendations resulting from our independent audits of the government’s programs and its 
Crown agencies and corporations. In doing so, the Office assists the Assembly in holding the 
government and its administrators accountable for the quality of the administration's 
stewardship of public funds and for the achievement of value for money in government 
operations. 












INDEPENDENCE 


The Provincial Auditor is appointed as an officer of the Legislative Assembly by the 
Lieutenant Governor in Council on the address of the Assembly. This is done after 
consultation with the Chair of the Standing Committee on Public Accounts (for more 
information on the Committee, see Chapter Seven). The Provincial Auditor and staff of the 
Office are independent of the government and its administration. This independence is a 
safeguard that enables the Office to fulfill its auditing and reporting responsibilities 
objectively and fairly. We are entitled to have access to all relevant information and records 
necessary to the performance of our duties under the Audit Act. 


The Board of Internal Economy—an all-party legislative committee that is independent of 
the government's administrative process—treviews and approves our budget, which is 
subsequently laid before the Legislative Assembly. As required by the Audit Act, the Office’s 
expenditures relating to the 2001/02 fiscal year have been audited by a firm of chartered 
accountants appointed by the Board and are presented at the end of this chapter. The 
audited statement of expenditure is submitted annually to the Board and subsequently 
tabled in the Legislative Assembly. 


AUDIT RESPONSIBILITIES 


We audit the financial statements of the province and the accounts of many agencies of the 
Crown, and we audit the administration of government programs and activities, as carried 
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out by ministries and agencies of the Crown under government policies and legislation. Our 
responsibilities are set out in the Audit Act (see Exhibit Four). 


The Office reports on its audits in an Annual Report to the Legislative Assembly. In 
addition, the Office may make a special report to the Assembly at any time on any matter 
that in the opinion of the Provincial Auditor should not be deferred until the Annual 
Report. We also assist and advise the Standing Committee on Public Accounts in its review 
of the Annual Report of the Provincial Auditor and of the Public Accounts of the province. 


It should be noted that our audit activities include the examination of the application of the 





government's administrative policies and the actual administration of the government’s 
policy decisions as carried out by ministry or agency management. Out of respect for the 
principle of Cabinet privilege, the Office does not seek access to the deliberations of 
Cabinet. However, the Office can access all other information contained in Cabinet 





submissions that we deem necessary to fulfill our auditing and reporting responsibilities 
under the Audit Act. 


/ Chapter Six 


The government is held accountable for policy matters by the Legislative Assembly, which 
continually monitors and challenges government policies and programs through questions 
during legislative sessions and through reviews of legislation and expenditure estimates. 


ACCOUNTS OF THE PROVINCE AND VALUE FOR 
MONEY 


The Provincial Auditor, under subsection 9(1) of the Audit Act, is required to audit the 
accounts and records of the receipt and disbursement of public money forming part of the 
Consolidated Revenue Fund whether held in trust or otherwise. To this end, the Office 
carries out an annual attest audit to enable the Provincial Auditor to express an opinion on 
the province's financial statements. As well, the Office carries out cyclical value-for-money 
audits in accordance with subsection 12(2) of the Audit Act. 


AGENCIES OF THE CROWN AND CROWN- 
CONTROLLED CORPORATIONS 


The Provincial Auditor, under subsection 9(2) of the Audit Act, is required to audit those 
agencies of the Crown that are not audited by another auditor. Exhibit One, Part (1), lists 
the agencies that were audited during the 2001/02 audit year. Public accounting firms are 
currently contracted by the Office to audit the financial statements of several of these 
agencies on its behalf. 


Exhibit One, Part (II) and Exhibit Two list the agencies of the Crown and Crown- 
controlled corporations, respectively, that were audited by public accounting firms during 
the 2001/02 audit year. Subsection 9(2) of the Audit Act requires public accounting firms 


that are appointed auditors of certain agencies of the Crown to perform their audits under 
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the direction of, and to report to, the Provincial Auditor. Under subsection 9(3) of the Act, 
public accounting firms auditing Crown-controlled corporations are required to deliver to 
the Provincial Auditor a copy of the audited financial statements of the corporation and a 
copy of their report of their findings and recommendations to management (management 
letter). 


ADDITIONAL RESPONSIBILITIES 


Under section 16 of the Audit Act, the Provincial Auditor may, by resolution of the Standing 
Committee on Public Accounts, be required to examine and report on any matter 
respecting the Public Accounts. 


Section 17 of the Act requires the Provincial Auditor to undertake special assignments 
requested by the Assembly, the Standing Committee on Public Accounts (by resolution of 
the Committee), or by a minister of the Crown. However, these special assignments are not 
to take precedence over the Provincial Auditor’s other duties. The Provincial Auditor can 
decline an assignment referred by a minister if, in the opinion of the Provincial Auditor, it 
conflicts with other duties. 


During the period of audit activity covered by this report (October 2001 to September 
2002), the Provincial Auditor was involved in the following special assignments: 


¢ On October 12, 2000, the Standing Committee on Public Accounts passed the 
following motion: 


The Public Accounts Committee directs that at such time as the OEB [Ontario Energy 
Board] and the Canadian Nuclear Safety Commission have completed their review of the 
leasing deal of Bruce Nuclear A and B and the agreement is completed, the Provincial 
Auditor examine all details of the leasing agreement between Ontario Power Generation 
and Bruce Partnership for the Bruce A and B nuclear facilities, to determine if the deal 
offers value for money for Ontario taxpayers, and to report to the Public Accounts 
Committee with his findings and recommendations as soon as possible. 

¢ On February 28, 2001, the Standing Committee on Public Accounts passed the 

following motion: 

That the Provincial Auditor, under section 17 of the Audit Act, be asked to conduct a value- 
for-money audit of the policy decision by Cancer Care Ontario to provide after-hours 
radiation therapy through a private clinic rather than in-house and report back to the 
Public Accounts Committee as soon as possible. 

The reports on the above special assignments were submitted to the Committee on June 6, 

2002 and December 13, 2001, respectively. 


In addition, on June 18, 2002, the Provincial Auditor accepted a special assignment from 
the Minister of Health and Long-Term Care to verify Ontario’s indicators of health. The 
assignment followed from the 2000 First Ministers’ Meeting that issued a Communiqué on 
Health that directed health ministers to provide comprehensive and regular public 
reporting on their health programs and services and on their respective health system’s 
performance. They also were to collaborate on the development of a comprehensive 
framework using comparable indicators. As part of this process, each jurisdiction was to 
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determine appropriate third-party verification for itself. Our Office agreed to perform 
specified procedures to provide assurance to the public regarding the reliability of Ontario’s 
reported indicators. The results of our work were reported to the Minister of Health and 
Long-Term Care. 


AUDIT ACTIVITIES 
TYPES OF AUDITS 


Value-for-money, attest, and compliance audits are the three main types of audits carried 
out by the Office. The Office generally conducts compliance audit work as a component of 
its value-for-money and attest audits. In addition, inspection audits of selected grant- 
recipient organizations may be conducted under section 13 of the Audit Act. The following 
are brief descriptions of each of these audit types. 


Value-for-Money Audits 


Subsection 12(2) of the Audit Act requires the Office to report on any cases observed where 
money was spent without due regard for economy and efficiency, or where appropriate 
procedures were not in place to measure and report on the effectiveness of programs. This 
value-for-money mandate is exercised through the auditing of various ministry and Crown- 
agency programs and activities each year. We have summarized in Chapter Three the 
conclusions, observations, and recommendations arising from the value-for-money work we 


performed between October 2001 and September 2002. 


It is not part of the Office’s mandate to measure, evaluate, or report on the effectiveness of 
programs or to develop performance measures or standards. These functions are the 
responsibility of ministry and/or agency management. The Office is responsible for 
reporting on whether or not ministry or agency management has carried out these functions 
satisfactorily. Our value-for-money work deals with the administration of programs by 
management, including major information systems. 


We plan, perform, and report our value-for-money work in accordance with the 
professional standards for assurance engagements, encompassing value for money and 
compliance, established by the Canadian Institute of Chartered Accountants. These 
standards require that we employ rigorous processes to maintain the quality, integrity, and 
value of our work for our client, the Legislative Assembly. They also require that we clearly 
explain the nature and extent of the assurance provided as a result of our work. Some of 
these processes and the degree of assurance they enable us to provide are described below. 


SELECTION OF PROGRAMS AND ACTIVITIES FOR AUDIT 


Major ministry and agency programs and activities are audited at approximately five-year 
intervals. Various factors are considered in selecting programs and activities for audit each 
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year. These factors include: the results of previous audits; the total revenues or expenditures 
at risk; the impact of the program or activity on the public; the inherent risk due to the 
complexity and diversity of operations; the significance of possible issues that may be 
identified by an audit; and the costs of performing the audit in relation to the perceived 
benefits. Possible issues are identified primarily through a preliminary survey of the program 
or agency. 

We also consider the work completed or planned by ministry and agency internal auditors. 
The relevance, timeliness, and breadth of scope of the work done by internal auditors can 
have a major impact on the timing, frequency, and extent of our audits. By having access to 
internal audit work plans, working papers, and reports and by relying, to the extent possible, 
on internal audit activities, the Office is able to avoid duplication of effort. 


OBJECTIVES AND ASSURANCE LEVELS 


The objective of our value-for-money work is to meet the requirements of subsection 12(2) 
of the Audit Act, which is to identify and report significant value-for-money issues. We also 
include in our Annual Report recommendations for improving controls, obtaining better 
value for money, and achieving legislated objectives. Management responses to each of these 
recommendations are reproduced in the Annual Report. 


The specific objective(s) for each audit or review conducted are clearly stated in the 
Objective(s) and Scope section of each audit report—that is, each value-for-money section 
of Chapter Three. Our work is designed to allow us to conclude on our stated objective(s). 


In almost all cases, our work is planned and performed to provide an audit level of 
assurance. An audit level of assurance is obtained by: interviewing management and 
analyzing the information they provide; examining and testing systems, procedures, and 
transactions; confirming facts with independent sources; and, where necessary, obtaining 
expert assistance and advice in highly technical areas. 


An audit level of assurance refers to the highest reasonable level of assurance the Office can 
provide concerning the subject matter. Absolute assurance that all significant matters have 
been identified is not attainable for various reasons, including: the use of testing; the 
inherent limitations of control; the fact that much of the evidence available is persuasive 
rather than conclusive in nature; and the need to exercise professional judgment. 


Infrequently, for reasons such as the nature of the program or activity, limitations in the 
Audit Act, or the prohibitive cost of providing a high level of assurance, the Office will 
perform a review rather than an audit. A review provides a moderate level of assurance 
because it consists primarily of inquiries and discussions with management; analyses of 
information they provide; and only limited examination and testing of systems, procedures, 
and transactions. For example, our work reported in Chapter Four of this report—Follow- 


up of Recommendations in the Special Report on Accountability and Value for Money 
(2000)—falls into this review category. 


eee 
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CRITERIA 


In accordance with professional standards for assurance engagements, work is planned and 
performed to provide a conclusion on the objective(s) set for the work. A conclusion is 
reached and observations and recommendations made by evaluating the administration of a 
program or activity against suitable criteria. Suitable criteria are identified at the planning 
stage of our audit or review by performing extensive research of sources, such as: recognized 
bodies of experts; applicable laws, regulations, and other authorities; other bodies or 
jurisdictions delivering similar programs and services; management’s own policies and 
procedures; and applicable criteria successfully applied in other audits or reviews. 


To further ensure their suitability, the criteria being applied are fully discussed with and 
agreed to by senior management responsible for the program or activity at the planning 
stage of the audit or review. 


COMMUNICATION WITH SENIOR MINISTRY OR AGENCY MANAGEMENT 


To help ensure the factual accuracy of our observations and conclusions, staff from our 
Office maintain ongoing communication with senior management throughout the audit or 
review. Before beginning the work, our staff meet with them to discuss the objectives and 
criteria and the focus of our work in general terms. During the audit or review, our staff 
meet with management to review progress and ensure open lines of communication. At the 
conclusion of on-site work, management is briefed on the preliminary results of the work. A 
draft report is then prepared and discussed with them. Management provides written 
responses to our recommendations and these are discussed and incorporated into the final 
draft report. The Provincial Auditor discusses and finalizes the draft report, on which the 
Annual Report will be based, with the deputy minister or agency head well in advance of 
the publication of the Annual Report. 


Attest Audits 


Attest (financial) audits are designed to permit the expression of the auditor's opinion on a 
set of financial statements in accordance with generally accepted auditing standards. The 
opinion states whether the operations and financial position of the entity have been fairly 
presented in compliance with appropriate accounting policies. ‘The Office conducts attest 
audits of the financial statements of the province and of numerous Crown agencies on an 
annual basis. 


Compliance Audits 


Subsection 12(2) of the Audit Act also requires the Office to report observed instances 
where: 


* accounts were not properly kept or public money was not fully accounted for; 


° essential records were not maintained or the rules and procedures applied were not 
sufficient to safeguard and control public property or to effectively check the assessment, 
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collection, and proper allocation of revenue or to ensure that expenditures were made 
only as authorized; or 


* money was expended other than for the purposes for which it was appropriated. 
Accordingly, as part of our value-for-money work, we: 


¢ identify provisions in legislation and authorities that govern the programs or agencies 
being examined or that the management of those programs or agencies is responsible for 
administering; and 


¢ perform such tests and procedures as we deem necessary to obtain reasonable assurance 
that management has complied with legislation and authorities in all significant respects. 


Inspection Audits of 
Grant-recipient Organizations 


Although grants to organizations such as hospitals, universities, community colleges, school 
boards, and thousands of smaller organizations amount to approximately 50% of total 
government expenditures, they are subject to only limited-scope inspection audits. 
Inspection audits are defined in the Audit Act as an examination of accounting records. 
Although value-for-money observations may arise as a by-product of inspection audits, the 
audits are not value-for-money oriented because only accounting records can be examined 
in inspection audits. 


The Office may, where circumstances warrant the extension of a ministry or agency audit, 
conduct inspection audits of grant recipients. In the past, the Office has carried out 
inspection audits of major recipients of grants, specifically community colleges, universities, 
hospitals, and school boards. However, in recent years, the Office has deferred major 
inspection-audit activity pending consideration of a proposal to amend the Audit Act to 
permit the Office to access all records and information necessary to perform full-scope 
audits, including value-for-money, of grant recipients. 


On April 19, 2001, the government announced in the Speech from the Throne, under the 
heading of “Holding the Broad Public Sector Accountable to Taxpayers,” that it would be 
introducing sweeping reforms to ensure that all public-sector institutions are accountable to 
the citizens of Ontario. Included in the planned reforms announced in the Throne Speech 
was a commitment to make amendments to the Audit Act that would permit the Provincial 
Auditor to assess the extent to which institutions funded by Ontario taxpayers use that 
money prudently, effectively, and as intended. The government's intention in this regard 
was also referred to in Budget Paper F of the 2001 Ontario Budget. Further details and 
background on the subject of amendments to the Audit Act are provided in Chapter Two, 
in the section entitled “Legislative Proposals to Improve Public-Sector Accountability.” 


Payments are also made to individuals under a variety of programs, such as the Ontario 


Health Insurance Plan or the Ontario Disability Support program. Such individual 
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recipients of government funds are not, and should not be, subject to direct audit by the 
Provincial Auditor. For these kinds of programs, our audits focus on the ministries’ 
procedures to ensure that only eligible recipients are being paid the correct amount. 


REPORTING ACTIVITIES 
VALUE-FOR-MONEY AUDITS 


As each audit or review is completed, the Office prepares a preliminary draft report for 
discussion and factual clearance. The preliminary draft report is discussed with senior 
ministry or agency officials and revised, as necessary, to reflect the results of the discussion. 
The Provincial Auditor discusses and finalizes the draft report with the deputy minister or 
agency head (chair) well in advance of the publication of the Annual Report. Following 
clearance of the preliminary draft report and the ministry or agency response, a final draft 
report is prepared and issued to the deputy minister or agency head and, where deemed 
necessary, to the minister. These final-draft audit reports form the basis of our Annual 
Report to the Legislative Assembly. 


AGENCY ATTEST AUDITS 


With respect to reporting on attest audits of agencies, agency legislation normally stipulates 
that the Provincial Auditor's reporting responsibilities are to the agency’s board and the 
minister(s) responsible. Also, we provide copies of the audit opinions and of the related 
agency financial statements to the deputy minister of the associated ministry, as well as to the 
Secretary of the Management Board of Cabinet. 


In instances where matters that require improvements by management have been noted 
during the course of an agency attest audit, a draft management letter is prepared, discussed 
with senior management, and revised as necessary to reflect the results of the discussion. 
Following clearance of the draft management letter and the response of the agency’s senior 
management, a final management letter is prepared and, if deemed necessary, issued to the 
agency head. Depending on the significance of the content of the management letter, a 
copy of it may also be forwarded to the minister and deputy minister of the associated 
ministry and to the Secretary of the Management Board of Cabinet. Matters of significance 
contained in the management letter may also be included in the Provincial Auditor’s 


Annual Report to the Legislative Assembly. 


CONFIDENTIALITY OF WORKING PAPERS 


In the course of our reporting activities, we prepare draft audit reports and management 
letters that are considered to be an integral part of our audit working papers. It should be 
noted that these working papers, according to section 19 of the Audit Act, are not required 
to be laid before the Assembly or any of its committees. 
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SPECIAL ASSIGNMENTS 


Under sections 16 and 17 of the Audit Act, the Office has additional reporting 
responsibilities relating to special assignments for the Legislative Assembly, the Standing 
Committee on Public Accounts, or a minister of the Crown. At the conclusion of such 
work, the Provincial Auditor normally reports to the initiating authority of the assignment. 


ANNUAL REPORT 


The reporting requirements for the Annual Report, as specified in subsection 12(2) of the 
Audit Act, are organized in the following way: 


Chapter One of the Annual Report provides an overview of the Provincial Auditor’s 
findings for this year’s value-for-money audit activities as well as summaries of the value-for- 
money audits and reviews that were conducted. 


Chapter Two contains observations on the subject of improving public-sector accountability. 


Chapter Three contains the reports resulting from our value-for-money audits of ministries 
and agencies conducted during the year. 


Chapter Four contains the results of our follow-up on the action taken by each ministry and 
agency on the recommendations we made to them two years ago. We conduct this follow- 
up to encourage timely attention by each ministry and agency to our observations and 
recommendations. Accordingly, a detailed account of the current status of 
recommendations made in the Special Report on Accountability and Value for Money (2000) 
is provided in this chapter. 


Chapter Five is devoted to the Provincial Auditor's comments on the audit of the Public 
Accounts of the province. The reporting requirements under subsections 12(2)(d) and (e) of 
the Audit Act are also met in this chapter. 


In Chapter Six, we report on the activities of the Office of the Provincial Auditor and 


reproduce the Office’s externally audited statement of expenditure for the year ended 
March 31, 2002. 


Chapter Seven provides information on the composition and activities of the Standing 
Committee on Public Accounts. 


OFFICE ORGANIZATION AND 
PERSONNEL 


The Office is organized into portfolio teams—a framework that attempts to align somewhat 
related audit entities and to foster expertise in the various areas of audit activity. [he 
portfolios, which are loosely based on the government’s own ministry organization, are each 
headed by a Director who oversees and is responsible for the ministry and agency audits 
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within the assigned portfolio. Assisting the portfolio Directors and rounding out the teams 
are a number of audit Managers and various other audit staff. 


The Provincial Auditor, the Assistant Provincial Auditor, and the portfolio Directors make 
up the Office’s Executive Steering Committee (ESC). The executive management of the 
Office as at September 30, 2002 is outlined in the following table. 


Executive Management of the Office of the Provincial Auditor, 2002 


Erik Peters Provincial Auditor 
Jim McCarter Assistant Provincial Auditor 





Paul Amodeo Public Accounts; Finance; Information Technology; and 
Accounting Research 


Walter Bordne Community, Family and Children’s Services; and Revenue 
Andrew Cheung Justice and Regulatory 
Gerard Fitzmaurice Economic Development 


John McDowell Crown Agencies, Corporations, Boards, and 
Commissions; and Transportation 


Nick Mishchenko Health and Long-Term Care; and Management Board 
Secretariat 


Gary Peall Education; Training, Colleges and Universities; Culture; 
Municipal Affairs and Housing; and Professional 
Practices 





Note: Annemarie Wiebe, the Manager of Human Resources, regularly attends meetings of the 
ESC to provide advice on matters related to human resources. 


The audit managers, together with the members of the ESC, constitute the Office’s 


Resource Planning and Allocation Committee. 


CODE OF PROFESSIONAL CONDUCT 


The Office has a Code of Professional Conduct to encourage staff to maintain high 
professional standards and ensure a professional work environment. It is intended to be a 
general statement of philosophy, principles, and rules regarding conduct for employees of 
the Office, who have a duty to conduct themselves in a professional manner and to strive to 
achieve the highest standards of behaviour, competence, and integrity in their work. The 
Code provides the reasoning for these expectations and further describes the Office’s 
responsibilities to the Legislative Assembly, the public, and our audit entities. The Code also 
provides guidance on disclosure requirements and the steps to be taken to avoid conflict-of- 
interest situations. 
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CANADIAN COUNCIL 
OF LEGISLATIVE AUDITORS 


The 30th annual meeting of the Canadian Council of Legislative Auditors (CCOLA) was 
held in St. John’s, Newfoundland from August 25 to 27, 2002. This annual gathering, 
bringing together legislative auditors from the federal government and the provinces, 
provides a useful forum for sharing ideas and exchanging information important to the 
work of the legislative auditing community. 


The Provincial Auditor and the Assistant Provincial Auditor attended this year’s meeting, 
which covered such topics as: 


¢ Accountabilities of Legislative Auditors 
e Performance Reporting 


e Performance Indicators for Health Care 


ACKNOWLEDGEMENT 
AUDITEES AND STAFF 


The Provincial Auditor expresses sincere appreciation to the officials of ministries, agencies, 
and other entities for their co-operation in providing his staff with all the information and 
explanations required during the performance of the Office’s audit work. 


The Provincial Auditor extends a special appreciation to the staff of the Office for their 
dedication, competence, and the professional manner in which they have carried out their 
duties. 


OFFICE EXPENDITURE 


The following is the 2002 audited Statement of Expenditure for the Office. 
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Salaries and wages 

Employee benefits (note 3) 
Transportation and communication 
Services 


Supplies and equipment 


Transfer payment: 
CCAF-FCVI Inc. 


The Audit Act (statutory) (note 4 





Office of the Provincial Auditor of Ontario 
Statement of Expenditure 
Year Ended March 31, 2002 
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Notes to Statement of Expenditure: 


1. 


Accounting Policy 


The statement of expenditure has been prepared using the prescribed Ministry of 
Finance modified cash basis of accounting, which allows for an additional 30 
days to pay for goods and services received during the fiscal year just ended. 


Estimates 


The “Estimates” shown on the statement of expenditure reflect the approved 
Expenditure Estimates of the Province of Ontario for the respective years. 


Employee Benefits 


The Office provides pension benefits for eligible employees through participation 
in the Public Service Pension Fund (PSPF), a multiemployer pension plan 
established by the Province of Ontario. The plan is accounted for as a 
multiemployer defined contribution pension plan. As such, the Office’s cash 
contribution to the PSPF for the year of $190,869 (2001 — $259,891) reflects the 
Office’s pension expenditures and is included in employee benefits. 


The cost of post-retirement non-pension benefits is paid by Management Board 
Secretariat and therefore is not included in this statement. 


Statutory Appropriations 


Under section 5 of the Audit Act, the Provincial Auditor's remuneration for the 
fiscal year is considered a statutory item. As well, under section 23 of the Act, 
specialist assistance of $97,370 incurred in connection with the Office’s 
examination of the Bruce Nuclear leasing agreement as requested by the 
Standing Committee on Public Accounts is considered a statutory item. 
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The actual 2001 expenditure includes approximately $88,000 for retroactive 
remuneration for the years from 1996 to 2000 as approved by the Board of 
Internal Economy in accordance with section 5 of the Audit Act. 


5. Public Sector Salary Disclosure Act, 1996 


Section 3(5) of this Act requires disclosure of Ontario public-sector employees 
paid an annual salary in excess of $100,000 in calendar year 2001. For the 
Office, this disclosure is as follows: 













Salary —_—— Taxable 
_ Paid Benefits 


208,841 20,209 
140,794 
110957, 
LO Sy, 


110,957 
110,957 255 
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AUDITORS’ REPORT TO THE BOARD OF INTERNAL ECONOMY 
OF THE LEGISLATIVE ASSEMBLY OF ONTARIO 


We have audited the statement of expenditure of the Office of the Provincial Auditor for 
the year ended March 31, 2002. This statement is the responsibility of the management of 
the Office of the Provincial Auditor. Our responsibility is to express an opinion on this 
statement based on our audit. 


We conducted our audit in accordance with Canadian generally accepted auditing 
standards. Those standards require that we plan and perform an audit to obtain reasonable 
assurance whether the statement is free of material misstatement. An audit includes 
examining, on a test basis, evidence supporting the amounts and disclosures in the 
statement. An audit also includes assessing the accounting principles used as well as 
evaluating the overall statement presentation. 


In our opinion, this statement presents fairly, in all material respects, the expenditures of the 
Office of the Provincial Auditor for the year ended March 31, 2002 in accordance with the 


accounting policy referred to in note 1 to the statement. 


Toronto, Canada ALLEN & MILES LLP 
July 10, 2002 CHARTERED ACCOUNTANTS 
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CHAPTER SEVEN 


The Standing Committee 
on Public Accounts 





APPOINTMENT AND COMPOSITION OF 
THE COMMITTEE 


The Standing Orders of the Legislature provide for the appointment of an all-party 
Standing Committee on Public Accounts. The Committee is appointed for the duration of 
the Parliament (that is, the period from the opening of the first session immediately 
following a general election to the end of a government's term and the calling of another 
election). 


The membership of the Committee reflects proportionately the representation of parties in 
the Legislature. All members are entitled to vote on motions with the exception of the Chair, 
whose vote is restricted to the breaking of a tie. 


In accordance with the Standing Orders, a Standing Committee on Public Accounts was 
appointed on November 1, 1999, soon after the commencement of the First Session of the 
Thirty-seventh Parliament. The membership of the Committee at September 30, 2002 was 
as follows: 


John Gerretsen, Chair, Liberal 

Bruce Crozier, Vice-chair, Liberal 

Steve Gilchrist, Progressive Conservative 
Raminder Gill, Progressive Conservative 
John Hastings, Progressive Conservative 
Shelley Martel, New Democrat 

Al McDonald, Progressive Conservative 
Richard Patten, Liberal 


ROLE OF THE COMMITTEE 


The Committee examines, assesses, and reports to the Legislature on a number of issues, 
including the economy and efficiency of government operations; the effectiveness of 
programs in achieving their objectives; controls over assets, expenditures, and the assessment 
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and collection of revenues; and the reliability and appropriateness of information in the 
Public Accounts. 


In fulfilling this role, pursuant to its terms of reference in the Standing Orders of the 
Assembly, the Committee reviews the Provincial Auditor’s Annual Report and the Public 
Accounts and reports to the Legislature its observations, opinions, and recommendations. 
Under the Standing Orders, the documents are deemed to have been permanently referred 
to the Committee as they become available. 


PROVINCIAL AUDITOR’S ROLE WITH THE 
COMMITTEE 


In accordance with section 16 of the Audit Act, the Provincial Auditor and senior staff 
attend committee meetings at which the Committee reviews the Provincial Auditor's Annual 
Report and the Public Accounts and assist the Committee in planning its agenda. 


COMMITTEE PROCEDURES AND 
OPERATIONS 


GENERAL 


The Committee meets on Thursday mornings when the Legislature is sitting. At times, the 
Committee also meets during the summer and winter when the Legislature is not sitting. All 
meetings are open to the public with the exception of those dealing with the setting of the 
Committee’s agenda and the preparation of committee reports. All public Committee 
proceedings are recorded in Hansard (the official verbatim report of the debates in the 
House, speeches, other proceedings in the Legislature, and all open-session sittings of 
standing and select committees). 


The Committee selects matters from the Provincial Auditor’s Annual Report and the Public 
Accounts for hearings. The Committee then requests senior ministry and agency officials to 
appear and respond to questions at these hearings. Since the Committee is concerned with 
administrative rather than policy matters, ministers rarely attend. 


Once hearings are completed, the Committee then reports its comments and 
recommendations to the Legislature. Committee procedures include the following: 


¢ in-depth briefings and preparation; 
* when practical, the inclusion of ministry responses in committee reports; and 
° follow-up of committee recommendations. 


The Committee also follows up in writing with those ministries and Crown agencies not 
selected for detailed review by the Committee regarding their plans and timetables for 


eee 
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addressing the concerns raised in the Provincial Auditor’s reports. This process enables each 
auditee to update the Committee on activities undertaken since the completion of the audit, 
such as any initiatives taken to address the Provincial Auditor’s recommendations. 


MEETINGS HELD 


From October 2001 to September 2002, the Committee met regularly on Thursday 
mornings when the Legislature was sitting and also met during the winter recess to consider 
the reports of the Provincial Auditor. The Committee’s work during this period relating to 
the work of this Office included: 


* reviewing the following items from the Provincial Auditor’s 2001 Annual Report and 
2000 Special Report on Accountability and Value for Money: 


2001 Annual Report 
- Ministry of Agriculture, Food and Rural Affairs—Food Industry Program; 


- Ministries of the Attorney General, Correctional Services, and Solicitor General 
(now the ministries of the Attorney General and of Public Safety and Security)— 
Integrated Justice Project; 


- Ministry of Community and Social Services (now the Ministry of Community, 
Family and Children’s Services) —Violence Against Women Program; 


- Ministry of Finance—Community Reinvestment Fund; 

- Ontario Innovation Trust; 

- Ministry of Transportation—Road User Safety Program; 

- the following follow-ups of recommendations contained in the 1999 Annual Report: 


2 Ministry of Economic Development and Trade (now the Ministry of Enterprise, 
Opportunity, and Innovation)—Financial Control Review; 


2 Ministry of Finance—Provincial Personal Income Tax Revenue and Related 
Credits and Reductions; and 


2 Ministry of Health and Long-Term Care—Ontario Substance Abuse Bureau; 
2000 Special Report 


- Ministry of Consumer and Commercial Relations (now the Ministry of Consumer 
and Business Services)—Project to Automate the Land Registration System 


(POLARIS); 
- Ministry of the Environment—Operations Division; 
* reviewing the Provincial Auditor’s report, Special Audit of Cancer Care Ontario; 


* reviewing the Provincial Auditor’s report, Special Audit of the Bruce Nuclear Transaction; 
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° — discussing the Office’s proposed amendments to the Audit Act, and 
e finalizing reports to the Legislature. 


During the period from October 2001 to September 2002, the Committee held hearings 
on the following Bills referred by the House: 


¢ Ethics and Transparency in Public Matters Act, 2002 (Bill 95); and 
© Public Sector Employee's Severance Pay Disclosure Act, 2002 (Bill 53). 


The House has also referred the Audit Amendment Act, 2002 (Bill 5) to the Committee for 
detailed review. It is expected that the Committee will hold hearings on Bill 5 in the fall 
2002 session. 


REQUESTS FOR SPECIAL AUDITS 


The Standing Committee on Public Accounts passed the following two motions on 
October 12, 2000 and February 28, 2001, respectively: 
1. The Public Accounts Committee directs that at such time as the OEB [Ontario Energy 
Board] and the Canadian Nuclear Safety Commission have completed their review of 
the leasing deal of Bruce Nuclear A and B and the agreement is completed, the 
Provincial Auditor examine all details of the leasing agreement between Ontario 
Power Generation (OPG) and Bruce Partnership for the Bruce A and B nuclear 
facilities, to determine if the deal offers value for money for Ontario taxpayers, and to 


report back to the Public Accounts Committee with his findings and recommendations 
as soon as possible. 


2. That the Provincial Auditor, under section 17 of the Audit Act, be asked to conduct a 
value-for-money audit of the policy decision by Cancer Care Ontario to provide 
after-hours radiation therapy through a private clinic rather than in-house and report 
back to the Public Accounts Committee as soon as possible. 
The reports on the above special assignments were submitted to the Committee on June 6, 


2002 and December 13, 2001, respectively, at which times the reports were discussed. 


REPORTS OF THE COMMITTEE 
GENERAL 


The Committee issues its reports to the Legislature. These reports contain a précis of the 


information reviewed by the Committee during its meetings, together with comments and 
recommendations. 


All committee reports are available through the Clerk of the Committee, thus enabling 
public access to full details of committee deliberations. 


After the Committee tables its report in the Legislative Assembly, it requests that ministries or 


agencies respond within 120 days or within time frames stipulated by the Committee in its 
individual recommendations. 
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During the period from October 2001 to September 2002, the Committee submitted the 
following reports to the Legislative Assembly: 


° Operations Division, Ministry of the Environment; and 


° POLARIS (Ministry of Consumer and Business Services). 


FOLLOW-UP OF RECOMMENDATIONS MADE BY 
THE COMMITTEE 


The Clerk of the Committee is responsible for following up on the actions taken on the 
Committee's recommendations by ministries or agencies. The Office of the Provincial 
Auditor confers with the Clerk to ascertain the status of the recommendations and, if 
considered necessary, brings any significant matters to the attention of the Legislature 


through the Provincial Auditor's Annual Report. 


Detailed information on the Committee’s recommendation respecting amendments to the 
Audit Act and other related matters is contained in Chapter Two of this report. 


OTHER COMMITTEE ACTIVITIES 


Canadian Council of Public Accounts Committees 
(CCPAC) 


CCPAC consists of delegates from federal, provincial, and territorial public accounts 
committees from across Canada. CCPAC meets at the same time and place as the Canadian 
Council of Legislative Auditors (CCOLA) to discuss issues of current interest. The twenty-third 
annual meeting of CCPAC was held in St. John’s, Newfoundland from August 25 to 27, 2002. 
The annual CCPAC and CCOLA meetings also permit the delegations to participate in a joint 
session to discuss subjects of mutual interest to politicians and legislative auditors. The 2002 joint 
session with CCOLA was on the subject of “Measuring Progress: The State of Federal 


Government Finances.” 
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EXHIBIT ONE 


Agencies of the Crown 


AgriCorp 


Algonquin Forestry Authority 

Cancer Care Ontario 

Centennial Centre of Science and Technology 

Chief Election Officer, Election Finances Act 

Election Fees and Expenses, Election Act 

Financial Services Commission of Ontario 

Grain Financial Protection Board, Funds for Producers of Grain Corn, Soybeans and Canola 
Investor Education Fund, Ontario Securities Commission 

Legal Aid Ontario 

Liquor Control Board of Ontario 

Livestock Financial Protection Board, Fund for Livestock Producers 
Northern Ontario Heritage Fund Corporation 

Office of the Assembly 

Office of the Environmental Commissioner 

Office of the Information and Privacy Commissioner 

Office of the Children’s Lawyer 

Office of the Ombudsman 

Ontario Clean Water Agency (December 31) 

Ontario Development Corporation 

Ontario Educational Communications Authority 

Ontario Electricity Financial Corporation 

Ontario Exports Inc. 

Ontario Financing Authority 

Ontario Food Terminal Board 

Ontario Heritage Foundation 

Ontario Housing Corporation (December 31) 

Ontario Immigrant Investor Corporation 

Ontario Media Development Corporation 

Ontario Northland Transportation Commission (December 31) 
Ontario Place Corporation 

Ontario Racing Commission 

Ontario Realty Corporation 

Ontario Securities Commission 

Ontario SuperBuild Corporation 

Ontario Tourism Marketing Partnership Corporation 

Province of Ontario Council for the Arts 

Provincial Judges Pension Fund, Provincial Judges Pension Board 
Public Guardian and Trustee for the Province of Ontario 
Toronto Area Transit Operating Authority 

TVOntario Foundation 


Agencies of the Crown 
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Exhibit One 


Agencies whose accounts are audited by anoth au ditor 
_ under the direction of the Provincial Au _ 


Board of Ciamene Mental Health Clinic, Guelph 
Niagara Parks Commission (October 31) 

Ontario Mental Health Foundation 

St. Lawrence Parks Commission 


Workplace Safety and Insurance Board (December 31) 





Notes: 
1. Dates in parentheses indicate fiscal periods ending on a date other than March 31. 


2. Changes during the 2001/02 fiscal year: 


Additions: 
* Ontario SuperBuild Corporation 


Inactive: 
« North Pickering Development Corporation 
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EXHIBIT TWO 


Crown-controlled 
Corporations 











Access Centre for Community Care in Lanark, Leeds and Grenville 
Access Centre for Hastings and Prince Edward Counties 
Algoma Community Care Access Centre 

Art Gallery of Ontario Crown Foundation 

Baycrest Hospital Crown Foundation 

Big Thunder Sports Park Ltd. 

Board of Funeral Services 

Brant Community Care Access Centre 

Brock University Foundation 

Carleton University Foundation 

CIAR Foundation (Canadian Institute for Advanced Research) 
Canadian Opera Company Crown Foundation 












Canadian Stage Company Crown Foundation 

Chatham/Kent Community Care Access Centre 

Cochrane District Community Care Access Centre 

Community Care Access Centre (CCAC) — Oxford 
Community Care Access Centre for Huron 

Community Care Access Centre for Kenora and Rainy River Districts 
Community Care Access Centre for the Eastern Counties 
Community Care Access Centre Niagara 

Community Care Access Centre of Halton 

Community Care Access Centre of London 

Community Care Access Centre of Peel 

Community Care Access Centre of Waterloo Region 
Community Care Access Centre Wellington-Dufferin 
Community Care Access Centre of York Region 

Community Care Access Centre Perth County 

Community Care Access Centre Simcoe County 

Community Care Access Centre Timiskaming 

Community Care Access Centre of The District of Thunder Bay 





















Dairy Farmers of Ontario 

Deposit Insurance Corporation of Ontario 
Durham Access to Care 

East York Access Centre for Community Services 
Education Quality and Accountability Office 
Elgin Community Care Access Centre 
Etobicoke Community Care Access Centre 
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_ Corporations whose accounts are audited by an auditor other than the Provincial - 
1 wih full access by the Provincial PNT eli Col at ce) Buen her: mere sia dba . 
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(continued) 





Foundation at Queen’s University at Kingston 
Grand River Hospital Crown Foundation 
Greater Toronto Transit Authority 
Grey-Bruce Community Care Access Centre 
Haldimand-Norfolk Community Care Access Centre 

Haliburton, Northumberland and Victoria Long-Term Care Access Centre 
Hamilton Community Care Access Centre 

Hydro One Inc. 

Kingston, Frontenac, Lennox and Addington Community Care Access Centre 
Lakehead University Foundation 

Laurentian University of Sudbury Foundation 

Manitoulin-Sudbury Community Care Access Centre 

McMaster University Foundation 

McMichael Canadian Art Collection 

Metropolitan Toronto Convention Centre Corporation 

Mount Sinai Hospital Crown Foundation 

National Ballet of Canada Crown Foundation 

Near North Community Care Access Centre 

Nipissing University Foundation 

North York Community Care Access Centre 

North York General Hospital Crown Foundation 

Ontario Family Health Network 

Ontario Foundation for the Arts 





Ontario Lottery and Gaming Corporation 
Ontario Mortgage Corporation 

Ontario Municipal Employees Retirement Board 
Ontario Pension Board 

Ontario Power Generation Inc. 

Ontario Trillium Foundation 

Ottawa Community Care Access Centre 

Ottawa Congress Centre 

Renfrew County Community Care Access Centre 
Royal Botanical Gardens Crown Foundation 
Royal Ontario Museum 

Royal Ontario Museum Crown Foundation 
Ryerson Polytechnic University Foundation 
Sarnia/Lambton Community Care Access Centre 
Scarborough Community Care Access Centre 
Science North 

Shaw Festival Crown Foundation 

Smart Systems for Health Agency 

St. Clair Parks Commission 

St. Michael’s Hospital Crown Foundation 
Stadium Corporation of Ontario Limited 
Stratford Festival Crown Foundation 
Sunnybrook Hospital Crown Foundation 
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Corporations whose accounts are audited by an auditor other than the Provincial 


Auditor, with full access by the Provincial Auditor to audit reports, working papers, 
and other related documents 
(continued) 





The Peterborough Community Access Centre Incorporated 
Toronto Community Care Access Centre 
Toronto Hospital Crown Foundation 
| Toronto Islands Residential Community Trust Corporation 
Toronto Symphony Orchestra Crown Foundation 
Trent University Foundation 
University of Guelph Foundation 
University of Ottawa Foundation 
University of Toronto Foundation 
University of Waterloo Foundation 
University of Western Ontario Foundation 
University of Windsor Foundation 
Waterfront Regeneration Trust Agency 
Wilfrid Laurier University Foundation 
Windsor/Essex Community Care Access Centre 
Women’s College and Wellesley Central Crown Foundation 
York Community Care Access Centre 
York University Foundation 


Notes: 
Changes during the 2001/02 fiscal year: 
Additions: 
* Access Centre for Community Care in Lanark, Leeds and Grenville 
* Access Centre for Hastings and Prince Edward Counties 
= Algoma Community Care Access Centre 
= Brant Community Care Access Centre 
«= _Chatham/Kent Community Care Access Centre 
* Cochrane District Community Care Access Centre 
«= Community Care Access Centre (CCAC) — Oxford 
«= Community Care Access Centre for Huron 
* Community Care Access Centre for Kenora and Rainy River Districts 
* Community Care Access Centre for the Eastern Counties 
* Community Care Access Centre Niagara 
= Community Care Access Centre of Halton 
= Community Care Access Centre of London 
«=  Cormmunity Care Access Centre of Peel 
* Community Care Access Centre of Waterloo Region 
* Community Care Access Centre Wellington-Dufferin 
* Community Care Access Centre of York Region 
* Community Care Access Centre Perth County 
* Community Care Access Centre Simcoe County 
« Community Care Access Centre Timiskaming 
* Community Care Access Centre of The District of Thunder Bay 
= Durham Access to Care 
* East York Access Centre for Community Services 
* Elgin Community Care Access Centre 
* Etobicoke Community Care Access Centre 
* Greater Toronto Transit Authority 
* Grey-Bruce Community Care Access Centre 
«= _Haldimand-Norfolk Community Care Access Centre 
* Haliburton, Northumberland and Victoria Long-Term Care Access Centre 
= Hamilton Community Care Access Centre 


Notes continued... 
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Notes (continued): 
Changes during the 2001/02 fiscal year: 
Additions (continued): 
» Kingston, Frontenac, Lennox and Addington Community Care Access Centre 
» —Manitoulin-Sudbury Community Care Access Centre 
» Near North Community Care Access Centre 
» North York Community Care Access Centre 
» Ottawa Community Care Access Centre 
= Renfrew County Community Care Access Centre 
« Sarnia/Lambton Community Care Access Centre 
« Scarborough Community Care Access Centre 
» Smart Systems for Health Agency 
= The Peterborough Community Access Centre Incorporated 
« Toronto Community Care Access Centre 
» Windsor/Essex Community Care Access Centre 
«York Community Care Access Centre 
Deletions: 
« _Moosonee Development Area Board 
» Ontario SuperBuild Corporation 
» Toronto East General Hospital Crown Foundation 
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EXHIBIT THREE 


Treasury Board Orders 





Under sub-section 12(2)(e) of the Audit Act, the Provincial Auditor is required to annually 
report all orders of the Management Board of Cabinet authorizing payments in excess of 
appropriations, stating the date of each order, the amount authorized, and the amount 
expended. ‘These are outlined in the following table. 


Amounts Authorized and Expended Thereunder Year Ended March 31, 2002 








Agriculture, Food and Rural Affairs Mar. 5, 2002 402,000 — 











Attorney General Jan. 15, 2002 5,800,000 3,665,241 













Feb. 26, 2002 4,052,300 2,428,300 
Mar. 5, 2002 11,213,400 2,242,199 
Mar. 6, 2002 4,220,100 3,901,568 

25,285,800 12,237,308 














Citizenship Mar. 11, 2002 810,000 == 











Community and Social Services Jan. 15, 2002 47,800,000 47,800,000 
Mar. 5, 2002 51,529,800 20,170,805 
99,329,800 67,970,805 



















Consumer and Business Services Feb. 5, 2002 4,804,000 4,657,919 
Mar. 5, 2002 7,512,500 3,066,760 
12,316,500 7,724,679 




















Correctional Services Mar. 5, 2002 41,685,000 41,288,549 











Economic Development and Trade Mar. 5, 2002 2,425,000 1,223,404 












Education Feb. 6, 2002 4,262,500 1,623,759 
Mar. 5, 2002 4,727,400 476,826 
8,989,900 2,100,585 





















Energy, Science and Technology Feb. 5, 2002 3,600,000 2,586,097 
Feb. 26, 2002 1,630,000 729,613 
5,230,000 3,315,710 




















Environment rep. 6. 2002 8,310,000 8,310,000 












Mar. 6, 2002 3,200,000 3,108,686 
Mar. 19, 2002 8,048,300 4,822,401 
19,558,300 16,241,087 















Finance Feb. 26, 2002 29,923,000 2,240,353 
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‘Date of Order 








Office of Francophone Affairs Feb. 12, 2002 72,000 














Health and Long-Term Care Mar. 11, 2002 310,257,900 258,149,259 












Labour Jan. 15, 2002 747,100 747,100 
Feb. 26, 2002 alten OO 425,217 
1,458,200 11/2047 



























Office of the Lieutenant Governor Feb. 5, 2002 104,200 89,861 





















Management Board Secretariat Dec. 6, 2001 9,000,000 9,000,000 
Mar. 5, 2002 26,932,400 16,862,378 
Mar. 19, 2002 98,756,600 - 88,714,352 





134,689,000 114,576,730 














Municipal Affairs and Housing June 14, 2001 1,375,000 1,213,666 
Dec. 11, 2001 2,872,800 2,872,800 
Dec. 11, 2001 2,121,700 1,685,922 
Feb. 5, 2002 15,000,000 15,000,000 
Mar. 5, 2002 19,976,300 14,754,938 





41,351,800 35,527,326 











Natural Resources Aug. 7, 2001 8,000,000 8,000,000 
Aug. 28, 2001 15,000,000 15,000,000 
Dec. 6, 2001 10,000,000 10,000,000 
Mar. 5, 2002 = 2 107,100 978,170 





35,707,100 33,978,170 








Northern Development and Mines Feb. 12, 2002 1,318,600 199,603 












Solicitor General Mar. 5, 2002 26,976,500 21,358,857 
Mar. 6, 2002 1,590,000 1,584,939 
28,566,500 22,943,796 























Tourism, Culture and Recreation Jan. 15, 2002 13,000,000 13,000,000 
Mar. 19, 2002 9,005,400 7,935,497 
Apr. 3, 2002 1,450,000 730,947 





23,455,400 21,666,444 


















Training, Colleges and Universities Mar. 5, 2002 





12,968,000 TAUESSE 

















Transportation Feb. 6, 2002 41,513,000 Sfarovita 
Mar. 5, 2002 20,594,200 19,660,570 
Mar. 5, 2002 9,656,700 1,325,466 


71,763,900 58,364,149 











_TOTAL TREASURY BOARD ORDERS 907,667,900 708,186,072 
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EXHIBIT FOUR 


The Audit Act 
R.S.O. 1990, Chapter A.35 





Amended by: 1999, c. 5, s. 1; 1999, c. 11. 
Definitions 1. In this Act, 


“agency of the Crown” means an association, authority, board, commis- 
sion, corporation, council, foundation, institution, organization or 


other body, 


(a) whose accounts the Auditor is appointed to audit by its sharehold- 
ers or by its board of management, board of directors or other 
governing body, 


(b) whose accounts are audited by the Auditor under any other Act or 
whose accounts the Auditor is appointed by the Lieutenant Gover- 
nor in Council to audit, 


(c) whose accounts are audited by an auditor, other than the Auditor, 
appointed by the Lieutenant Governor in Council, or 


(d) the audit of the accounts of which the Auditor is required to direct 
or review or in respect of which the auditor’s report and the 
working papers used in the preparation of the auditor’s statement 
are required to be made available to the Auditor under any other 


Act, 


but does not include one that the Crown Agency Act states is not 
affected by that Act or that any other Act states is not a Crown agency 
within the meaning or for the purposes of the Crown Agency Act; 
(“organisme de la Couronne”) 


“Assistant Auditor” means the Assistant Provincial Auditor; (“Vérificateur 
adjoint”) 
“Auditor” means the Provincial Auditor; (“Vérificateur”) 


“Board” means the Board of Internal Economy referred to in section 87 
of the Legislative Assembly Act, (“Commission”) 
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Office of the Auditor 


Provincial Auditor 


Tenure of office and 
removal 


Salary of Auditor 


Same 


Idem 





“Crown controlled corporation” means a corporation that is not an 
agency of the Crown and having 50 per cent or more of its issued and 
outstanding shares vested in Her Majesty in right of Ontario or having 
the appointment of a majority of its board of directors made or ap- 
proved by the Lieutenant Governor in Council; (“société contrélée par 
la Couronne’) 


“fiscal year” has the same meaning as in the Ministry of Treasury and 
Economics Act, (“exercice”) 


“inspection audit” means an examination of accounting records; 
(“vérification”) 
“Office of the Auditor” means the Office of the Provincial Auditor; 


(“Bureau du Vérificateur’) 


“public money” has the same meaning as in the Financial Administration 


Act. (“deniers publics”) R.S.O. 1990, c. A.35,s. 1. 
2. The Office of the Provincial Auditor shall consist of the Auditor, 


the Assistant Auditor and such employees as may be required from time 
to time for the proper conduct of the business of the Office. 
R907 1990 oo). 


3. The Auditor shall be appointed as an officer of the Assembly by the 
Lieutenant Governor in Council on the address of the Assembly after 
consultation with the chair of the standing Public Accounts Committee 


of the Assembly. R.S.O. 1990, c. A.35, s. 3. 
4, The Auditor may hold office until the end of the month in which 


he or she attains the age of sixty-five years and may be reappointed for a 
period not exceeding one year at a time until the end of the month in 
which he or she attains seventy years of age, but is removable at any time 
for cause by the Lieutenant Governor in Council on the address of the 


Assembly, R.S.O. 1990, cxA:35, 5-4. 
5.—(1) The Auditor shall be paid a salary within the highest range of 


salaries paid to deputy ministers in the Ontario civil service and is entitled 
to the privileges of office of a senior deputy minister. R.S.O. 1990, 
C295, S: FL) L990 Ves se) Ch) 0 some inC ii 


(2) The salary of the Auditor, within the salary range referred to in 
subsection (1), shall be determined and reviewed annually by the Board. 
1999 eens hy): 


(3) The salary of the Auditor shall be charged to and paid out of the 
Consolidated Revenue Fund. R.S.O. 1990, c. A.35, s. 5 (3). 
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Appointment of 
Assistant Auditor 


Duties of Assistant 
Auditor 


Qualifications 


Audit of 
Consolidated 
Revenue Fund 


Audit of agencies of 
the Crown 


Audit of Crown 
controlled 
corporations 





The Audit Act 


6. The Assistant Auditor shall be appointed as an officer of the Assem- 
bly by the Lieutenant Governor in Council upon the recommendation 


of the Auditor. R.S.O. 1990, c. A.35, s. 6. 
7. The Assistant Auditor, under the direction of the Auditor, shall 


assist in the exercise of the powers and the performance of the duties of 
the Auditor and, in the absence or inability to act of the Auditor, shall act 
in the place of the Auditor. R.S.O. 1990, c. A.35, s. 7. 


8. The persons appointed as Auditor and Assistant Auditor shall be 
persons who are licensed under the Public Accountancy Act. 


Ris: © M99 0eAS > su8: 
9.—(1) The Auditor shall audit, on behalf of the Assembly and in 


such manner as the Auditor considers necessary, the accounts and 
records of the receipt and disbursement of public money forming part of 
the Consolidated Revenue Fund whether held in trust or otherwise. 
Reo O90 FCrARS 5) seo! (1): 


(2) Where the accounts and financial transactions of an agency of the 
Crown are not audited by another auditor, the Auditor shall perform the 
audit, and, despite any other Act, where the accounts and financial 
transactions of an agency of the Crown are audited by another auditor, 
the audit shall be performed under the direction of the Auditor and such 
other auditor shall report to the Auditor. R.S.O. 1990, c. A.35, s. 9 QO) 


(3) Where the accounts of a Crown controlled corporation are 
audited other than by the Auditor, the person or persons performing the 
audit, 


(a) shall deliver to the Auditor forthwith after completion of the audit 
a copy of their report of their findings and their recommendations 
to the management and a copy of the audited financial statements 
of the corporation; 


(b) shall make available forthwith to the Auditor, when so requested 
by the Auditor, all working papers, reports, schedules and other 
documents in respect of the audit or in respect of any other audit 
of the corporation specified in the request; 


(c) shall provide forthwith to the Auditor, when so requested by the 
Auditor, a full explanation of work performed, tests and 
examinations made and the results obtained, and any other 
information within the knowledge of such person or persons in 


respect of the corporation. R.S.O. 1990, c. A.35, s. 9 (3). 
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Information and 
access to records 


Accommodation in 
ministries and 
Crown agencies 


Annual report 


Contents of report 


(4) Where the Auditor is of the opinion that any information, 
explanation or document that is provided, made available or delivered to 
him or her by the auditor or auditors referred to in subsection (2) or (3) 
is insufficient, the Auditor may conduct or cause to be conducted such 
additional examination and investigation of the records and operations of 
the agency or corporation as the Auditor considers necessary. 


RUSK O MMR Oasis We) <n Cle 


10. Every ministry of the public service, every agency of the Crown 
and every Crown controlled corporation shall furnish the Auditor with 
such information regarding its powers, duties, activities, organization, 
financial transactions and methods of business as the Auditor from time 
to time requires, and the Auditor shall be given access to all books, 
accounts, financial records, reports, files and all other papers, things or 
property belonging to or in use by the ministry, agency of the Crown or 
Crown controlled corporation and necessary to the performance of the 


duties of the Auditor under this Act. R.S.O. 1990, c. A.35, s. 10. 


11. For the purposes of exercising powers or performing duties under 
this Act, the Auditor may station one or more members of the Office of 
the Auditor in any ministry of the public service, in any agency of the 
Crown and in any Crown controlled corporation and the ministry, 
agency or corporation shall provide such accommodation as is required 


for such: purposes. (R:S.0,, 1990, c.7.3555 i 


12.—(1) The Auditor shall report annually to the Speaker of the 
Assembly after each fiscal year is closed and the Public Accounts are laid 
before the Assembly, but not later than the 31st day of December in each 
year unless the Public Accounts are not laid before the Assembly by that 
day, and may make a special report to the Speaker at any time on any 
matter that in the opinion of the Auditor should not be deferred until 
the annual report, and the Speaker shall lay each such report before the 
Assembly forthwith if it is in session or, if not, not later than the tenth day 
of the next session. R.S.O. 1990, c. A.35,s. 12 (1). 


(2) In the annual report in respect of each fiscal year, the Auditor shall 


report on, 


(a) the work of the Office of the Auditor, and on whether in carrying 
on the work of the Office the Auditor received all the information 
and explanations required; 


(b) the examination of accounts of receipts and disbursements of 
public money; 





458 


2002 Annual Report of the Provincial Auditor of Ontario 


SSS SSS SSS 


(0) 


(f) 





The Audit Act 


the examination of the statements of Assets and Liabilities, the 
Consolidated Revenue Fund and Revenue and Expenditure as 
reported in the Public Accounts, and shall express an opinion as to 
whether the statements present fairly the financial position of the 
Province, the results of its operations and the changes in its 
financial position in accordance with the accounting principles 
stated in the Public Accounts applied on a basis consistent with 
that of the preceding fiscal year together with any reservations the 
Auditor may have; 


all special warrants issued to authorize payments, stating the date of 
each special warrant, the amount authorized and the amount 


expended; 


all orders of the Management Board of Cabinet made to authorize 
payments in excess of appropriations, stating the date of each 
order, the amount authorized and the amount expended; 


such matters as, in the opinion of the Auditor, should be brought 
to the attention of the Assembly including, without limiting the 
generality of the foregoing, any matter related to the audit of 
agencies of the Crown or Crown controlled corporations or any 
cases where the Auditor has observed that, 


(i) accounts were not properly kept or public money was not 


fully accounted for, 


(i) essential records were not maintained or the rules and 
procedures applied were not sufficient to safeguard and 
control public property or to effectively check the assessment, 
collection and proper allocation of revenue or to ensure that 
expenditures were made only as authorized, 


(ii) money was expended other than for the purposes for which it 
was appropriated, 


(iv) money was expended without due regard to economy and 
efficiency, or 


(v) where procedures could be used to measure and report on 
the effectiveness of programs, the procedures were not 
established or, in the opinion of the Auditor, the established 
procedures were not satisfactory. R.S.O. 1990, c. A.35, 
oa 22). 
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Inspection audit 13.—(1) The Auditor may perform an inspection audit in respect of a 
payment in the form of a grant from the Consolidated Revenue Fund or 
an agency of the Crown and may require a recipient of such a payment 
to prepare and to submit to the Auditor a financial statement that sets out 
the details of the disposition of the payment by the recipient. 

Rs .OcLOD Ose AL secu, (ls 


Obstruction of (2) No person shall obstruct the Auditor or any member of the Office 
Auditor of the Auditor in the performance of an inspection audit or conceal or 
destroy any books, papers, documents or things relevant to the subject- 


matter of the inspection audit. R.S.O. 1990, c. A.35, s. 13 (2). 


Offence (3) Every person who knowingly contravenes subsection (2) and every 
director or officer of a corporation who knowingly concurs in such 
contravention is guilty of an offence and on conviction is liable to a fine of 
not more than $2,000 or to imprisonment for a term of not more than 


one year, or tapoth. Keo. Lo Waray sat (5) 


Idem, corporation (4) Where a corporation is convicted of an offence under subsection 
(3), the maximum penalty that may be imposed upon the corporation is 


$25,000 and not as provided therein. R.S.O. 1990, c. A.35, s. 13 (4). 


Examination onoath = 14, The Auditor may examine any person on oath on any matter 
pertinent to any account subject to audit by the Auditor or in respect of 
any inspection audit by the Auditor and for the purpose of such an 
examination the Auditor has the powers conferred upon a commission 
under Part I of the Public Inquiries Act, which Part applies to the exami- 
nation as if it were an inquiry under that Act. R.S.O. 1990, c. A.35, 

s. 14. 


Proviso 15. Nothing in this Act shall be construed to require the Auditor, 


(a) to report on any matter that, in the opinion of the Auditor, is 
immaterial or insignificant; or 


(b) to audit or direct the audit of or report on the accounts of a body 
not referred to in this Act in the absence of such a requirement in 


any other Act in respect of the body. R.S.O. 1990, c. A.35, s. 15. 


Attendance at 16. At the request of the standing Public Accounts Committee of the 


standing Public Assembly, the Auditor and any member of the Office of the Auditor 
Accounts Committee ; ; p : 
of the Asemnily designated by the Auditor shall attend at the meetings of the committee 


in order, 
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(a) to assist the committee in planning the agenda for review by the 
committee of the Public Accounts and the annual report of the 
Auditor; and 


(b) to assist the committee during its review of the Public Accounts 
and the annual report of the Auditor, 


and the Auditor shall examine into and report on any matter referred to 
him or her in respect of the Public Accounts by a resolution of the 
comimittice mbes OM9I0"csA Sox, 16; 


17. The Auditor shall perform such special assignments as may be 
required by the Assembly, the standing Public Accounts Committee of 
the Assembly, by resolution of the committee, or by a minister of the 
Crown in right of Ontario but such special assignments shall not take 
precedence over the other duties of the Auditor under this Act and the 
Auditor may decline an assignment by a minister of the Crown that, in 
the opinion of the Auditor, might conflict with the other duties of the 
Auditor R.S.O. 1990, c. A.35, s. 17. 


18. The Auditor may advise appropriate persons employed in the 
public service of Ontario as to any matter that comes or that may come to 
the attention of the Auditor in the course of exercising the powers or 


performing the duties of Auditor. R.S.O. 1990, c. A.35, s. 18. 


19. Audit working papers of the Office of the Auditor shall not be 
laid before the Assembly or any committee of the Assembly. 
Ree OL MEO ere Woon, Oy IGE 


20. Subject to the approval of the Board and to sections 22, 25 and 
26, the Auditor may employ such professional staff and other persons as 
the Auditor considers necessary for the efficient operation of the Office 
of the Auditor and may determine the salary of the Assistant Auditor and 
the salaries and remuneration, which shall be comparable to the salary 
ranges of similar positions or classifications in the public service of On- 
tario, and the terms and conditions of employment of the employees of 


the Office of the Auditor, R.S.O. 1990, c. A.35, s. 20. 


21.—(1) Every employee of the Office of the Auditor, before per- 
forming any duty as an employee of the Auditor, shall take and subscribe 
before the Auditor or a person designated in writing by the Auditor, 


(a) the following oath of office and secrecy, in English or in French: 
Jy Re saan EW! ca is are do swear (or solemnly affirm) that I will faithfully 


discharge my duties as an employee of the Provincial Auditor and will observe 
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and comply with the laws of Canada and Ontario and, except as I may be 
legally required, I will not disclose or give to any person any information or 
document that comes to my knowledge or possession by reason of my being 
an employee of the Office of the Auditor. 


So help me God. (Omit this line in an affirmation) 


(b) the following oath of allegiance, in English or in French: 


| rte ree eens ke Hy op Nie Ine Een hes , do swear (or solemnly affirm) that 
I will be faithful and bear true nllesanve to Her Majesty Queen Elizabeth the 
Second (or the reigning sovereign for the time being), her heirs and successors 
according to law. 


So help me God. (Omit this line in an affirmation) 


Rist 00nc eh 5 sa ah) 


(2) The Auditor may require any person or class of persons ap- 
pointed to assist the Auditor for a limited period of time or in respect of 
a particular matter to take and subscribe either or both of the oaths set 
out in subsection (1). R.S.O. 1990, c. A.35, s. 21 (2). 


(3) A copy of each oath administered to an employee of the Office 
of the Auditor under subsection (1) shall be kept in the file of the 
employee in the Office of the Auditor. R.S.O. 1990, c. A.35, s. 21 (3). 


(4) The failure of an employee of the Office of the Auditor to take 
and subscribe or to adhere to either of the oaths required by subsection 
(1) may be considered as cause for dismissal. R.S.O. 1990, c. A.35, 

s. 21 (4). 


22.—(1) The employee benefits applicable from time to time under 
the Public Service Act to civil servants who are not within a unit of 
employees established for collective bargaining under any Act apply or 
continue to apply, as the case may be, to the Auditor, the Assistant 
Auditor and to the full-time permanent and probationary employees of 
the Office of the Auditor and the Board or any person authorized by 
order of the Board may exercise the powers and duties of the Civil 
Service Commission and the Auditor or any person authorized in 
writing by the Auditor may exercise the powers and duties of a deputy 
minister under that Act in respect of such benefits. R.S.O. 1990, 
CONS, Sy oD), 


(2) The Auditor and the Assistant Auditor are members of the 
Public Service Pension Plan. R.S.O. 1990, c. A.35, s. 22 (2). 


23. Subject to the approval of the Board, the Auditor from time to 


time may appoint one or more persons having technical or special 


eee 
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knowledge of any kind to assist the Auditor for a limited period of time 
or in respect of a particular matter and the money required for the 
purposes of this section shall be charged to and paid out of the Consoli- 
dated Revenue Fund. R.S.O. 1990, c. A.35, s. 23. 


24. The Auditor may delegate in writing to any other member of the 
Office of the Auditor authority to exercise any power or perform any 
duty of the Auditor other than reporting to the Assembly. R.S.O. 1990, 
CoB suA: 


25.—(1) An employee of the Office of the Auditor shall not, 


(a) bea candidate in a provincial or federal election or in an election 
for any municipal office including a local board of a municipality 
within the meaning of the Municipal Affairs Act; 


(b) solicit funds for a provincial, federal or municipal party or candi- 


date; or 


(c) associate his or her position in the Office of the Auditor with any 
political activity. R.S.O. 1990, c. A.35, s. 25 (1). 


(2) Contravention of any of the provisions of subsection (1) may be 


considered as cause for dismissal. R.S.O. 1990, c. A.35, s. 25 (2). 


26.—(1) The Auditor may make orders and rules for the conduct of 
the internal business of the Office of the Auditor and, after a hearing, 
may suspend, demote or dismiss any employee of the Office of the 


Auditor for cause. R.S.O. 1990, c. A.35, s. 26 (1). 
(2) The Public Service Act and regulations thereunder that apply in 


relation to suspension from employment pending an investigation and in 
relation to a hearing by a deputy minister or his or her delegate as to 
cause for dismissal, other than as to notice to the Civil Service Commis- 
sion, apply with necessary modifications where the Auditor is of the 
opinion that there may exist cause for the suspension without pay, demo- 
tion or dismissal of an employee of the Office of the Auditor, and, for the 
purpose, the Auditor shall be deemed to be a deputy minister. 

RES ORTOOORC EAS), 72042)! 


(3) A decision of the Auditor to demote, suspend or dismiss an em- 
ployee may be appealed by the employee, within fourteen days after the 
decision has been communicated to the employee to the Public Service 
Grievance Board established under the Public Service Act. R.S.O. 1990, 
“saa a a aired ON Ges 
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(4) The Public Service Grievance Board may hear and dispose of an 
appeal under this section and the provisions of the regulation under the 
Public Service Act that apply in relation to a grievance for dismissal apply 
with necessary modifications to an appeal under this section, and, for the 
purpose, the Auditor shall be deemed to be a deputy minister and the 
decision of the Public Service Grievance Board is final and the Public 
Service Grievance Board shall report its decision and reasons in writing to 


the Auditor and to the appellant. R.S.O. 1990, c. A.35, s. 26 (4). 


27.—(1) No proceedings lie against the Auditor, the Assistant Audi- 
tor, any person employed in the Office of the Auditor or any person 
appointed to assist the Auditor for a limited period of time or in respect 
of a particular matter, for anything he or she may do or report or say in 
the course of the exercise or the intended exercise of functions under this 
Act, unless it is shown that he or she acted in bad faith. R.S.O. 1990, 
Cawaesez Uh): 


(2) The Auditor, the Assistant Auditor and each person employed in 
the Office of the Auditor or appointed to assist the Auditor for a limited 
period of time or in respect of a particular matter shall preserve secrecy 
with respect to all matters that come to his or her knowledge in the 
course of his or her employment or duties under this Act and shall not 
communicate any such matters to any person, except as may be required 
in connection with the administration of this Act or any proceedings 
under this Act or under the Criminal Code (Canada). R.S.O. 1990, 
Co se): 


28. A person or persons, not employed by the Crown or the Office of 
the Assembly, licensed under the Public Accountancy Act and appointed 
by the Board, shall examine the accounts relating to the disbursements of 
public money on behalf of the Office of the Auditor and shall report 
thereon to the Board and the chairman of the Board shall cause the 
report to be laid before the Assembly if it is in session or, if not, at the next 


session. R.S.O. 1990, c. A.35, s. 28. 


29.—(1) The Auditor shall present annually to the Board estimates of 
the sums of money that will be required for the purposes of this Act. 
Ro, Ono o0r es 355.201), 


(2) The Board shall review and may alter as it considers proper the 
estimates presented by the Auditor, and the chair of the Board shall cause 
the estimates as altered by the Board to be laid before the Assembly and 
the Assembly shall refer the estimates laid before it'to a committee of the 
Assembly for review. R.S.O. 1990, c. A.35, s. 29 (2). 


ee eee ee 
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(3) Notice of meetings of the Board to review or alter the estimates 
presented by the Auditor shall be given to the chair and the vice-chair of 
the standing Public Accounts Committee of the Assembly and the chair 
and the vice-chair may attend at the review of the estimates by the Board. 


Io: OF 1990 "e7555s, 29 (3). 


(4) ‘The money required for the purposes of this Act, other than 
under sections 5 and 23, shall be paid out of the money appropriated 
therefor by the Legislature. R.S.O. 1990, c. A.35, s. 29 (4). 
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